Settings for Crawling Another SharePoint Portal Server Site on the Internet

There may be instances when you want to expose your server, by using HTTP, to the Internet to allow other sites to crawl your server. The instructions in this section enable a SharePoint Portal Server computer with access to the Internet to crawl another SharePoint Portal Server computer across the Internet. Currently, SharePoint Portal Server only supports HTTP crawling. HTTPS crawling is not supported.

SharePoint Portal Server cannot use the default content access account to access another SharePoint Portal Server over the Internet. You can only specify an NTLM trusted domain account as the default content access account, and you cannot use NTLM for Internet crawls. You must specify an access account for the site path to the SharePoint Portal Server you want to crawl.

If the workspace you want to crawl has enabled Anonymous access (the Internet Guest Account is assigned to the reader role on the workspace), you do not need to use Basic authentication and do not need to perform the following procedure. You need only create a content source to the workspace on the other server. For information about assigning the Internet Guest Account to the reader role on a workspace, see the section, Security Settings earlier in this chapter.

To configure the server to crawl another SharePoint Portal Server computer across the Internet:

1. Navigate to the Content Sources folder located in the Management folder in the workspace.
2. Double-click Additional Settings.
3. On the Rules tab, click Site Paths.
4. Click New. The Create New Site Path Rule dialog box appears.
5. In Path, type the URL to the SharePoint Portal Server to be crawled. The URL must be the external FQDN of the server.
6. Click Include this path, and then click Options. The Options dialog box appears.
7. Click Account. The Account Information dialog box appears.
8. Under Account, specify the account information for the account that is valid for Basic authentication.
9. Under Authentication type, click Basic authentication (password is sent in clear text).
10. To close the Account Information dialog box, click OK, and then to close the Options dialog box, click OK.
11. To close the Create New Site Path Rule dialog box, click OK, and then, to close the Site Paths dialog box, click OK.
12. If prompted to start a full update, click Yes.
13. Click OK.

After completing the preceding steps, create a content source to the workspace on the SharePoint Portal Server computer that you want to crawl.