Skip to content

Releases: fortembr/semlinks-robots-parser

v0.0.11 NPM Ready

22 Oct 02:42

Choose a tag to compare

In the previous version, the array output could easily be logged to the console. However, it did not work well when imported into a project, and it didn't return the data correctly.

This version logs the array to the console, but also returns it as consumable data. This allows the front-end developers to display it in any way they choose.

Console.Log Output

 [
  {
    type: 'sitemap',
    content: 'https://carbondigital.us/sitemaps/post-sitemap1.xml'
  },
  {
    type: 'sitemap',
    content: 'https://carbondigital.us/sitemaps/page-sitemap1.xml'
  },
  {
    type: 'sitemap',
    content: 'https://carbondigital.us/sitemaps/projects-sitemap1.xml'
  },
  {
    type: 'sitemap',
    content: 'https://carbondigital.us/sitemaps/services-sitemap1.xml'
  },
  {
    type: 'sitemap',
    content: 'https://carbondigital.us/sitemaps/locations-sitemap1.xml'
  },
  {
    type: 'sitemap',
    content: 'https://carbondigital.us/sitemaps/category-sitemap1.xml'
  },
  { type: 'blank', content: '' },
  { type: 'user-agent', content: 'SemrushBot' },
  { type: 'disallow', content: '/' },
  { type: 'user-agent', content: 'SemrushBot-SA' },
  { type: 'disallow', content: '/' },
  { type: 'user-agent', content: 'MJ12bot' },
  { type: 'disallow', content: '/' },
  { type: 'user-agent', content: 'Mediapartners-Google' },
  { type: 'disallow', content: '' },
  { type: 'user-agent', content: 'Googlebot-Image' },
  { type: 'disallow', content: '' },
  { type: 'user-agent', content: '*' },
  { type: 'disallow', content: '/wp-admin/' },
  { type: 'allow', content: '/wp-admin/admin-ajax.php' }
]

Our SEM Links app is a Nest.js project. In that project, we have a data wrapper so all returned data is part of a "data" object. When testing this in Postman, here is the same data being returned.

{
    "data": [
        {
            "type": "sitemap",
            "content": "https://carbondigital.us/sitemaps/post-sitemap1.xml"
        },
        {
            "type": "sitemap",
            "content": "https://carbondigital.us/sitemaps/page-sitemap1.xml"
        },
        {
            "type": "sitemap",
            "content": "https://carbondigital.us/sitemaps/projects-sitemap1.xml"
        },
        {
            "type": "sitemap",
            "content": "https://carbondigital.us/sitemaps/services-sitemap1.xml"
        },
        {
            "type": "sitemap",
            "content": "https://carbondigital.us/sitemaps/locations-sitemap1.xml"
        },
        {
            "type": "sitemap",
            "content": "https://carbondigital.us/sitemaps/category-sitemap1.xml"
        },
        {
            "type": "blank",
            "content": ""
        },
        {
            "type": "user-agent",
            "content": "SemrushBot"
        },
        {
            "type": "disallow",
            "content": "/"
        },
        {
            "type": "user-agent",
            "content": "SemrushBot-SA"
        },
        {
            "type": "disallow",
            "content": "/"
        },
        {
            "type": "user-agent",
            "content": "MJ12bot"
        },
        {
            "type": "disallow",
            "content": "/"
        },
        {
            "type": "user-agent",
            "content": "Mediapartners-Google"
        },
        {
            "type": "disallow",
            "content": ""
        },
        {
            "type": "user-agent",
            "content": "Googlebot-Image"
        },
        {
            "type": "disallow",
            "content": ""
        },
        {
            "type": "user-agent",
            "content": "*"
        },
        {
            "type": "disallow",
            "content": "/wp-admin/"
        },
        {
            "type": "allow",
            "content": "/wp-admin/admin-ajax.php"
        }
    ]
}

v0.0.4 Initial PROD Release

20 Oct 04:49

Choose a tag to compare

We haven't been able to test this in our projects, but based on how it was developed, we are expecting it to work without fail. This will be confirmed in the next day or two. If there are any issues, we'll fix the bugs and release a new version immediately.

This includes the NPM release.