Activities

mankichi14

mankichi14 posted a new question

nuxt-generate-cluster •

Error: spawn E2BIG is occurred

Hi!
"Error: spawn E2BIG" is occurred when I do "nuxt-generate -w 4 -b"
I will generate 2864 routes.

Should I set other option?

midblue

midblue posted a new question

nuxt-generate-cluster •

@nuxt/sitemap not supported?

Hi! I'm using nuxt-generate-cluster to generate my static site (nuxt-generate -b).
Latest version of Nuxt, latest everything.
At the end of my generation, under normal circumstances (i.e. nuxt generate), my sitemap gets generated. Great!
However, with nuxt-generate-cluster, this doesn't seem to happen. Is there any way I can manually generate the sitemap, or add a hook the end of this other generate function?

nuxt.config.js:

...
modules: [
    'nuxt-i18n',
    '@nuxtjs/feed',
    '@nuxtjs/sitemap',
  ],
...
  generate: {
    dir: './docs',
    fallback: '404.html',
    workers: 4,
    workerConcurrency: 20,
    concurrency: 20,
    routes: () => {...}
  },
...
mtltechtemp

mtltechtemp posted a new question

nuxt-generate-cluster •

nuxt.config.ts not supported

Can you add support for TypeScript based Nuxt project please?

njam

njam posted a new feature request

nuxt-generate-cluster •

idea Easy way to make the CLI script fail on any errors

What problem does this feature solve?

When using nuxt-generate-cluster in CI it's crucial to detect errors to be able to abort the build process. Otherwise corrupt builds can get deployed.

Currently the CLI script exits with code 0 in many error cases, which makes it difficult to detect that something went wrong.

For example, when a worker process is killed due to hitting NodeJS's memory limit with signal SIGBART, the 'nuxt-generate' script exits with code 0.

$ NODE_OPTIONS=--max-old-space-size=200 nuxt-generate -w1
[...]
✔ generated: /de/events/basel/2019-01-26/5a97ec47-aef9-422c-a75f-db35429a6d84/index.html                                worker 1 18:50:44
✔ generated: /en/events/basel/2019-01-26/5a97ec47-aef9-422c-a75f-db35429a6d84/index.html                                worker 1 18:50:44
✔ generated: /fr/events/basel/2019-01-26/457c9d63-eec8-47b3-9c73-9ffcf5029797/index.html                                worker 1 18:50:44

<--- Last few GCs --->

[26057:0x55c3494ad3b0]     8054 ms: Mark-sweep 189.6 (213.7) -> 182.0 (214.2) MB, 35.8 / 0.0 ms  (average mu = 0.389, current mu = 0.391) allocation failure scavenge might not succeed
[26057:0x55c3494ad3b0]     8113 ms: Mark-sweep 189.8 (214.2) -> 182.2 (214.7) MB, 34.7 / 0.0 ms  (average mu = 0.396, current mu = 0.403) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1b9474ddbe1d]
Security context: 0x14ef8771e6e1 <JSObject>
    1: push [0x14ef87705851](this=0x024697cbac39 <JSArray[0]>,0x2534143843c9 <String[1]: ->)
    2: intoTokens [0x1f14ee734521] [/home/njam/Repos/denkmal-website/node_modules/clean-css/lib/tokenizer/tokenize.js:~80] [pc=0x1b94751efc0e](this=0x2290dd79ad11 <JSGlobal Object>,source=0x024697c85b41 <Very long string[1648]>,externalContext=0x024697c86261 <Object ...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x55c348323f21 node::Abort() [/usr/bin/node]
 2: 0x55c348323f6f  [/usr/bin/node]
 3: 0x55c3484d87f2 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/bin/node]
 4: 0x55c3484d8a4b v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/bin/node]
 5: 0x55c34886d283  [/usr/bin/node]
 6: 0x55c34886d3c4  [/usr/bin/node]
 7: 0x55c34887d043 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/bin/node]
 8: 0x55c34887d9c9 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/bin/node]
 9: 0x55c34887fadd v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/bin/node]
10: 0x55c34887fb32 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/bin/node]
11: 0x55c34884dcd5 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/bin/node]
12: 0x55c348ac028f v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/bin/node]
13: 0x1b9474ddbe1d
ℹ worker 1 exited with status code null by signal SIGABRT                                                                 master 18:50:46
HTML Files generated in 22.5s

$ echo $?
0

What does the proposed changes look like?

I would suggest to make the script exit with code 1 on:

  • Any worker exited with a code other than 0, or got killed
  • Any unhandled rendering errors

@pimlie wdyt?

I think the current behaviour (exit 0 even on critical errors) is quite unexpected. But changing it ofc is a breaking change. Not sure what your policy is?

If you think it makes sense, then I would be happy to prepare a PR

njam

njam posted a new question

nuxt-generate-cluster •

Argument 'workerInfo' on hook 'done' not passed

The third argument on the 'done' hook should be an Object called workerInfo, but actually I receive undefined.

Should be passed to done(workerInfo), but is not.

Version: 2.3.2

myelement88

myelement88 posted a new question

nuxt-generate-cluster •

Add Semantic-UI with jQuery components

Hello,

I'm trying to add Semantic-UI JS and jQuery via build script.

So far I was only able to add it via CDN. But the goal would be to add it via nuxt.config.js:

const pkg = require("./package"); const jqueryCDN = "https://cdnjs.cloudflare.com/ajax/libs/jquery/3.1.1/jquery.min.js"; const suiCDN = "https://cdn.jsdelivr.net/npm/[email protected]/dist/semantic.min.js"; import webpack from "webpack";
module.exports = { mode: "universal", head: { title: "Nuxt App", meta: [ { charset: "utf-8" }, { name: "viewport", content: "width=device-width, initial-scale=1" }, { hid: "description", name: "description", content: "pkg.description" } ], link: [{ rel: "icon", type: "image/x-icon", href: "/favicon.ico" }], script: [{ src: jqueryCDN }, { src: suiCDN }] }, loading: { color: "#0082b9" }, css: [ // Loads main style "~/assets/semantic/semantic.css", "~/assets/styles/main.css" ], modules: [], plugins: [], build: { // Jquery config over webpack plugins: new webpack.ProvidePlugin({ jQuery: "jQuery", $: "jQuery", "window.jQuery": "jQuery" }) } };

If I try to add Semantic-UI and jQuery via Plugins (node_modules):
{ src: "jquery" ssr: false } { src: "fomantic-ui" ssr: false }

I always get jQuery is Not Defined.

And of course i've studied the docs and the implementation of axios but that doesnt seem to work either.

Some help would be highly appreciated. Thanks in advance.

christophmayrhofer

christophmayrhofer posted a new question

nuxt-generate-cluster •

Get static routes automatically like with the regular nuxt generate command

I would like to use nuxt-generate as a replacement for nuxt generate

Unfortunately it generates 0 routes unless I manually define them in the generate.routes option

How can I have it automatically generate all routes based on the files in my pages folders the way that the original nuxt generate command does (it does not require me to define the routes in the nuxt.config.js file)

mvaneijgen

mvaneijgen posted a new question

nuxt-generate-cluster •

Setting base: in the generate: {} object

I am building an app to host on Github Pages so I changed my generate object in nux.config.js to

generate: {
    dir: 'docs',
    router: {
      base: '/<repo-name>/',
    },
  }

but it seems not possible to change the router with this when running npm run generate. What would be the correct way if setting this?

I still want to use npm run dev and have the app just run locally (localhost:3000) and use npm run generate and host it on Github Pages.

jhpmatos

jhpmatos posted a new question

nuxt-generate-cluster •

Task stops if callback is passed

When passing a callback to routes(callback, params). It works with only params: routes(params)

emdahlstrom

emdahlstrom posted a new question

nuxt-generate-cluster •

All workers complete their work and exit but build silently halts

I have trouble with the build/generation not finishing. The process seems to go fine and all workers complete their work and exit but then nothing happens. Maybe everything is done, just that npm doesn't exit the script.

Has anyone else run into this? Any tips for debugging this?

Here is the generate config from nuxt.config.js:

  generate: {
    workers: 4,
    workerConcurrency: 500,
    concurrency: 500,
    routes: function () {
      return axios.get('https://_-redacted-_/wp-json/wp/v2/posts?per_page=100')
      .then((res) => {
        return res.data.map((post) => {
          return '/posts/' + post.slug
        })
      })
    },
    done ({ duration, errors, workerInfo }) {
      if (errors.length) {
        console.log({ generate_errors: errors })
      }
    }
  },
pimlie

pimlie resolved the question #c1

nuxt-generate-cluster •

Should we rename this package

@Atinux
fyi, after careful consideration I changed my mind and dont think we should rename this package 👺

Using child_process directly will not give any major benefits above using the cluster api. And although it maybe feels/sounds a bit wrong to use cluster (at least to me) as thats primarly focussed on networking, there is nothing wrong with using it anyway: https://github.com/nodejs/help/issues/970

Also the name nuxt-generate-workers might be interpreted as that the package should be used next to another package (that provides the master) so lets keep it as it is

jameswragg

jameswragg posted a new question

nuxt-generate-cluster •

[Question] Generating static sites on-demand with Nuxt.js

Cross-post from nuxt.js/issues/#2370 but this repo also seems applicable…


Hi, I'm trying to create a static-site on-demand API using hapi.js to power the API & nuxt.js as the static-site generator but hitting some blockers….

Here's my setup:

  1. The API takes a POST payload
  2. Based on the payload the API dynamically creates the generate.routes array (passing payloads in for each route)
  3. Calls generate() to create the static site in ./dist

Issue:

This API is potentially going to be hammered - building a lot of individual content pages and pushing them to S3 buckets. All generated sites going to the ./dist directory is a problem - I currently have an isGenerating flag to prevent multiple generate calls, forcing the client to re-try the API.

  1. Is using Nuxt as a static-site generator on-demand a ridiculous idea?
  2. If (hopefully) not, are there ways of managing the output better. Potentially streaming straight to an S3 bucket?
  3. Should the generator be able to handle parallel builds?

Thanks in advance,
James

pimlie

pimlie posted a new question

nuxt-generate-cluster •

Should we rename this package

@Atinux @pi0

When I created this package I used the name cluster as I thought it didnt matter much as it would be a temporary package anyway. That would leave the option open for you to create a package with similar functionality but with a better name. Now that you accepted this package to the nuxt-community, could/should we maybe rename it to e.g. a more suitable nuxt-multi-generate or nuxt-generator-workers name? (better suggestions are welcome)

Also I implemented a cleaner and extensible producer/consumer design in my next branch. I am not sure yet if using cluster is the best in the long run, eg using process/child_process would be 'cleaner'.
Besides that I am thinking about creating a nuxt-generate-daemon that just runs all the time and will poll every interval the routes api endpoint to check for new routes/files to (re)generate. So instead of running this package in a crontab. As I will probably need different type of workers for that I am not sure yet cluster will be sufficient for that. With the new implementation we should be able to easily extend and implement process/subprocess if needed.

So maybe it would be an idea to remove the cluster part from this package name now the impact of that would be still small as there are not many users yet?