Infrastructure
  1. Infrastructure
  2. INFRA-1106

robots.txt on wiki.apache.org blocks Google (again)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: Initial Clearing
    • Component/s: MoinMoin
    • Labels:
      None

      Description

      A couple of years ago, https://issues.apache.org/jira/browse/INFRA-361 was marked "fixed", allowing Google to spider the wiki sites.

      However, while searching for some Spamassassin documnetation today, I noticed that virtually no wiki hits were appearing -- especially odd since our documentation and FAQ is primarily wiki-based.

      A check of http://wiki.apache.org/robots.txt reveals the cause:

      User-agent: *
      Disallow: /

      Can this be fixed? We rely on our user population being able to find documentation and FAQs via google. Should we move this from ASF infrastructure if there are persistent load problems?

        Activity

        Gavin made changes -
        Fix Version/s Initial Clearing [ 12325964 ]
        Tony Stevenson made changes -
        Workflow jira [ 12393954 ] INFRA Workflow [ 12715293 ]
        Henri Yandell made changes -
        Field Original Value New Value
        Resolution Fixed [ 1 ]
        Status Open [ 1 ] Closed [ 6 ]
        Justin Mason created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Justin Mason
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development