How can I refresh sitemap.xml in order to update lastmod of dynamic routes (data received from API)?

TitanFighter
15
TitanFighter
commented 2 months ago

I have sitemap config like this:

  sitemap: async function () {
    return {
      hostname: '',
      cacheTime: 1000, // also tried -1
      gzip: true,
      defaults: {
        priority: 1,
        changefreq: 'daily',
        lastmod: new Date()
      },
      routes: await sitemapDynamicPagesGet()
    }
  },

sitemapDynamicPagesGet returns array like this:

[
  {
    changefreq: 'daily',
    priority: 1,
    lastmod: '2020-12-30T17:18:03.566Z'
    url: 'http://localhost:3000/tour/name123'
  },
  {
    changefreq: 'daily',
    priority: 1,
    lastmod: '2021-01-08T19:31:33.636Z'
    url: 'http://localhost:3000/tour/name456'
  }
]

lastmod and url generated based on data received from API.

When I update/re-save tour details on the server (through admin page), I refresh Last edited datetime, but when I refresh sitemap.xml page in the browser I see old/previous lastmod (which was generated/received during "npm run build" process). I have tried to change cacheTime to -1 and to 1000, but this does not help.

I have read this from https://sitemap.nuxtjs.org/usage/sitemap-options#cachetime-optional---number:

Defines how frequently sitemap routes should be updated (value in milliseconds).
Setting a negative value will disable the cache.

Please note that after each invalidation, routes will be evaluated again (see routes declaration section).

and I understand it like in case of "cacheTime: 1000" the function "await sitemapDynamicPagesGet()" must be called, but it is not, so it looks like I do not understand what the cacheTime does/how it works.

What is the way to refresh/update lastmod? Is there a way to run "await sitemapDynamicPagesGet()" every X hours or on each sitemap.xml load in order to keep lastmod up to date?

Thanks a lot.

0
TitanFighter
15
TitanFighter
commented a month ago

Nico, thank you very much for your help!

I found my issue.

Incorrect code:

  sitemap: async function () {
    return {
      hostname: '',
      gzip: true,
      cacheTime: 1,
      defaults: {
        priority: 1,
        changefreq: 'daily',
        lastmod: new Date()
      },
      routes: await sitemapDynamicPagesGet()
    }
  },

Correct code:

  sitemap: {
    hostname: '',
    gzip: true,
    cacheTime: 1,
    defaults: {
      priority: 1,
      changefreq: 'daily',
      lastmod: new Date()
    },
    routes: async () => sitemapDynamicPagesGet()
  },

Just for reference (in case if someone needs it for any case), how my sitemapDynamicPagesGet() looks like:

export async function sitemapDynamicPagesGet () {
  return await Promise.all([
    function1(),
    function2()
  ])
    .then((results) => {
      const function1Data = results[0]
      const function2Data = results[1]

      const routes = []

      // Proceed function1Data and function2Data
      // ...
      xyz.forEach((data) => {
        routes.push({
          changefreq: 'daily',
          priority: 1,
          lastmod: data.editedAt,
          url: `http://example.com/${data.url}`
        })
      })

      return routes
    })
    .catch((error) => {
      console.log('sitemapDynamicPagesGet error:', error)
    })
}

Also useful example: https://sitemap.nuxtjs.org/usage/sitemap-options/#from-a-function-which-returns-a-promise

1
NicoPennec
2.2k
NicoPennec
commented 2 months ago

hi @TitanFighter,

The cache will be used on each /sitemap.xml call.

How to use the cachetime option:
cachetime = 0 => infinite cache (the cache will never expired)
cachetime = 1 => no cache (the cache will always be expired)
cachetime > 1 => cache time in milliseconds between 2 calls

Note that the cache will be initialized one time on nuxt start, then it's only on each call of the sitemap URL.

If you want a period refresh of cache, use an external way (ex. crontab + curl request)

0
TitanFighter
15
TitanFighter
commented 2 months ago

@NicoPennec thanks for your attention and time.

Updating sitemap.xml on each call is ok, but I still can not make it work.

In order to exclude some problems from my side I double checked with fresh nuxt installation and this is the nuxt.config.js:

export default {
  // Global page headers (https://go.nuxtjs.dev/config-head)
  head: {
    title: 'myproject',
    meta: [
      { charset: 'utf-8' },
      { name: 'viewport', content: 'width=device-width, initial-scale=1' },
      { hid: 'description', name: 'description', content: '' }
    ],
    link: [
      { rel: 'icon', type: 'image/x-icon', href: '/favicon.ico' }
    ]
  },

  // Global CSS (https://go.nuxtjs.dev/config-css)
  css: [
  ],

  // Plugins to run before rendering page (https://go.nuxtjs.dev/config-plugins)
  plugins: [
  ],

  // Auto import components (https://go.nuxtjs.dev/config-components)
  components: true,

  // Modules for dev and build (recommended) (https://go.nuxtjs.dev/config-modules)
  buildModules: [
  ],

  // Modules (https://go.nuxtjs.dev/config-modules)
  modules: [
    '@nuxtjs/sitemap'
  ],

  sitemap: {
    hostname: 'https://example.com',
    gzip: true,
    cacheTime: 1,
    defaults: {
      changefreq: 'daily',
      priority: 1,
      lastmod: new Date()
    },
    routes: [
      '/page/1',
      '/page/2',
      {
        url: '/page/3',
        changefreq: 'daily',
        priority: 1,
        lastmod: '2017-06-30T13:30:00.000Z'
      }
    ]
  },

  // Build Configuration (https://go.nuxtjs.dev/config-build)
  build: {
  }
}

It generates the next sitemap.xml:

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
    <url>
        <loc>https://example.com/page/1</loc>
        <lastmod>2021-01-09T20:46:02.667Z</lastmod>
        <changefreq>daily</changefreq>
        <priority>1.0</priority>
    </url>
    <url>
        <loc>https://example.com/page/2</loc>
        <lastmod>2021-01-09T20:46:02.667Z</lastmod>
        <changefreq>daily</changefreq>
        <priority>1.0</priority>
    </url>
    <url>
        <loc>https://example.com/page/3</loc>
        <lastmod>2017-06-30T13:30:00.000Z</lastmod>
        <changefreq>daily</changefreq>
        <priority>1.0</priority>
    </url>
    <url>
        <loc>https://example.com/</loc>
        <lastmod>2021-01-09T20:46:02.667Z</lastmod>
        <changefreq>daily</changefreq>
        <priority>1.0</priority>
    </url>
</urlset>

As you see I use cacheTime: 1 + lastmod: new Date(), so every time I open sitemap URL I must see new date, but I see the date generated during the "start" process.

What am I doing wrong?

0
NicoPennec
2.2k
NicoPennec
commented 2 months ago

@TitanFighter After checking, the defaults option is only evaluate by Nuxt on "nuxt start", so before the cache initialization…

As explained in the docs, the cacheTime option is only applied to the routes function evaluation: https://sitemap.nuxtjs.org/usage/sitemap-options#cachetime-optional---number

As workaround, you can use the filter option to generate a fresh lastmod value as below:

      filter: ({ routes }) => {
        return routes.map((route) => {
          if (!route.lastmod) {
            route.lastmod = new Date()
          }
          return route
        })
      },

See more about filter method on docs : https://sitemap.nuxtjs.org/usage/sitemap-options#filter-optional---function

0
TitanFighter
15
TitanFighter
commented a month ago

Nico, thank you very much for your help!

I found my issue.

Incorrect code:

  sitemap: async function () {
    return {
      hostname: '',
      gzip: true,
      cacheTime: 1,
      defaults: {
        priority: 1,
        changefreq: 'daily',
        lastmod: new Date()
      },
      routes: await sitemapDynamicPagesGet()
    }
  },

Correct code:

  sitemap: {
    hostname: '',
    gzip: true,
    cacheTime: 1,
    defaults: {
      priority: 1,
      changefreq: 'daily',
      lastmod: new Date()
    },
    routes: async () => sitemapDynamicPagesGet()
  },

Just for reference (in case if someone needs it for any case), how my sitemapDynamicPagesGet() looks like:

export async function sitemapDynamicPagesGet () {
  return await Promise.all([
    function1(),
    function2()
  ])
    .then((results) => {
      const function1Data = results[0]
      const function2Data = results[1]

      const routes = []

      // Proceed function1Data and function2Data
      // ...
      xyz.forEach((data) => {
        routes.push({
          changefreq: 'daily',
          priority: 1,
          lastmod: data.editedAt,
          url: `http://example.com/${data.url}`
        })
      })

      return routes
    })
    .catch((error) => {
      console.log('sitemapDynamicPagesGet error:', error)
    })
}

Also useful example: https://sitemap.nuxtjs.org/usage/sitemap-options/#from-a-function-which-returns-a-promise

1
Informations
QuestionResolved
#c127 - Created 2 months ago