Auto-check links in documentation

Automatically checking links in documentation ensures all hyperlinks are valid, functional, and not broken. This process improves user experience and SEO performance. Here’s how to implement auto-checking of links in documentation effectively:

1. Use Link Checker Tools

There are several tools available to automate link validation in documentation:

• Broken Link Checker (W3C)
  A web-based tool that checks for broken links on a page or across a website.

• Deadlink Checker
  Offers a simple online interface and can schedule regular checks.

• Dr. Link Check
  Scans entire websites for broken links with detailed reports.

• Xenu’s Link Sleuth (Windows)
  A free tool that crawls websites and reports broken links, redirects, and more.

• Screaming Frog SEO Spider
  A powerful desktop program that can crawl websites and identify broken links, among other SEO issues.

• Markdown Link Check
  A Node.js tool that scans .md files for broken links, ideal for GitHub-hosted documentation.

2. CI/CD Pipeline Integration

To ensure links are checked regularly and automatically:

• GitHub Actions:
  Use actions like lycheeverse/lychee-action or gaurav-nelson/github-action-markdown-link-check to check links on every commit or pull request.

• GitLab CI/CD:
  Add a job in .gitlab-ci.yml using markdown-link-check or lychee as a Docker job.

• CircleCI, Travis CI, Jenkins:
  Set up a job to run link-checking scripts as part of the documentation deployment pipeline.

Example GitHub Action Using Lychee:

yaml
CopyEdit
name: Link Checker

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  linkChecker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Check Links
        uses: lycheeverse/lychee-action@v1.4.1
        with:
          args: --verbose ./docs

3. Browser Extensions for Manual Checks

For on-the-fly manual validation during content editing:

• Check My Links (Chrome)
  Highlights valid and broken links in web pages quickly.

• LinkChecker (Firefox)
  An extension that scans the current page for HTTP response codes.

4. CMS Plugins for Link Validation

If you’re using a CMS like WordPress, Docusaurus, or Hugo:

• WordPress Plugins:

  • Broken Link Checker: Monitors your posts, pages, and comments for broken links.

  • SEOPress: Also checks for broken links among other SEO metrics.

• Docusaurus Plugin:
  Built-in link checker using yarn docusaurus check-links.

• Hugo:
  Use hugo --gc --minify and a post-build script to check links with tools like htmltest.

5. Best Practices for Maintaining Link Health

• Prefer Relative Links Internally
  Use relative URLs for internal navigation to reduce the risk of broken links when the domain changes.

• Redirect Strategies
  Maintain a 301 redirect list for moved resources to prevent 404 errors.

• Regular Audits
  Schedule link audits weekly or monthly, depending on update frequency.

• Avoid Deep Linking to Third-Party Pages
  Instead of linking directly to subpages on external sites, link to main pages or use reference names that are less likely to change.

6. Handling False Positives

Link checkers may flag temporary errors (e.g., rate-limited URLs, login-required pages). To manage these:

• Whitelist known problematic domains using tool-specific options.

• Retry failed links with delays before reporting.

• Use authenticated link checking where needed.

7. Link Validation in Markdown/HTML Files

To validate links in raw documentation files:

• Using markdown-link-check

  bash
  CopyEdit
  markdown-link-check README.md
  

• Using lychee (Rust-based, very fast)

  bash
  CopyEdit
  lychee docs/**/*.md
  

• Using htmltest for HTML output

  bash
  CopyEdit
  htmltest -c .htmltest.yml
  

8. Monitor with Analytics

Use tools like Google Analytics or Matomo to identify 404 pages visited by users. This reveals missed broken links.

Automated link checking in documentation significantly improves content quality, trustworthiness, and SEO. By integrating link validation into your development workflow, you ensure a seamless and error-free experience for readers. Regular monitoring and leveraging the right tools help maintain link health even as your content evolves.