Import website links

the most general type of content you can import is website links however, due to security and privacy/copyright concerns, not all links are allowed the base link must match your custom domain this feature will not work properly if you do not set a custom domain wildcards we also support links containing wildcards ` ` e g , https //www archbee com/blog/ this syntax will crawl all the links in https //www archbee com/blog/ and crunch the data limits the system supports a maximum of 500 links we will take the first 500 links passed, even those containing wildcard syntax sorting links links that are fed into the llm are sorted in the following order explicit links crawled links from wildcards make sure to pass your essential external resources as full links first if you are using wildcard syntax and know you might surpass the limit of 500 links