Upload and manage documents
Upload a document once, and use it with PageIndex Agent, Page Search and Page Extract. A document is one file, such as a PDF. It is made of pages, numbered from 1, and search results, citations and extractions all point back to them.
Set up your client as in the quick start first. With Page Search or Page Extract, use client.pageindex wherever these examples use client. For HTTP requests, see the API reference.
Upload
PageIndex takes PDFs, Word documents (.docx, .doc) and PowerPoint presentations (.pptx, .ppt). Page Extract takes PDFs only.
How you upload depends on the product you use:
| You use | Upload with | Price |
|---|---|---|
| PageIndex Agent | PageIndexClient.submit_document() | $1 per 1,000 pages |
| Page Search | PageSearchClient.upload() | $0.10 per 1,000 pages |
| Page Extract | PageExtractClient.extract(), which uploads the file for you | Free: you pay only for the extraction |
# wait=True waits until document preprocessing is ready
doc_id = client.submit_document("./report.pdf", wait=True)["doc_id"]
import { readFileSync } from "node:fs";
const { doc_id } = await client.api.submitDocument(readFileSync("./report.pdf"), "report.pdf");
await client.api.waitDocument(doc_id); // waits until document preprocessing is ready
The upload returns the document's ID. Keep it: every other call takes it.
This upload preprocesses the document for PageIndex Agent. If you only use Page Search, upload this way instead:
from pageindex import PageSearchClient
doc_id = PageSearchClient().upload("./report.pdf")["doc_id"] # waits until the PDF is ready to search
import { PageSearchClient } from "@pageindex/sdk";
const { doc_id } = await new PageSearchClient().upload({ path: "./report.pdf" }); // waits until the PDF is ready to search
A document uploaded for Page Search or Page Extract still works with PageIndex Agent, and you can build its outline later.
To put the document in a collection or a folder, see Collections or folders.
Document status
A document isn't ready the moment you upload it. It moves through these statuses:
| Status | What it means |
|---|---|
queued | Waiting to start. |
processing | PageIndex is reading the pages. |
completed | Ready to use. |
failed | It couldn't be read. |
The upload examples above wait until the document is completed. If it fails, the wait raises an error. The wait also gives up after 30 minutes, though the document goes on processing.
To upload without waiting in Python, leave out wait=True, then call client.wait_document(doc_id) when you need the document.
Build an outline later
A document uploaded for Page Search or Page Extract has no outline for PageIndex Agent to follow. You can build one later, at $1 per 1,000 pages, the same as an upload for PageIndex Agent.
client.build_outline_index(doc_id) # waits until the outline is built
await client.api.buildOutlineIndex(doc_id); // starts the build, which finishes in the background
Manage documents
Check a document
doc = client.get_document(doc_id)
print(doc["name"], doc["status"], doc["pageNum"])
const doc = await client.api.getDocument(doc_id);
console.log(doc.name, doc.status, doc.pageNum);
You get the document's name, status, page count, and its folder, collection and metadata if it has them. For what each status means, see Document status.
List documents
page = client.list_documents(limit=20, offset=0)
for doc in page["documents"]:
print(doc["id"], doc["name"], doc["status"])
print(page["total"]) # how many documents you have
const page = await client.api.listDocuments({ limit: 20, offset: 0 });
for (const doc of page.documents) {
console.log(doc.id, doc.name, doc.status);
}
Documents come newest first, 50 at a time unless you set limit.
To list only the documents in one collection or one folder, see Collections with metadata and List documents in a folder.
Read a document
Get the text of some pages, or a page as an image.
# The text of pages 5 to 7
for page in client.get_page_content(doc_id, "5-7"):
print(page["page_index"], page["markdown"][:200])
# Page 5 as an image, in JPEG bytes
image = client.page_image(doc_id, 5)
The JavaScript SDK reads pages through its agent tools. See Use with your agent.
Delete a document
client.delete_document(doc_id)
await client.api.deleteDocument(doc_id);
You can't delete a document while it is still queued or processing. Wait for it first.
Storage
Storing documents is free up to 10,000 pages. To store more, turn on usage-based billing in the PageIndex Console: storage past 10,000 pages then costs $0.10 per 1,000 pages a month. Without it, requests are refused once you hold more than 10,000 pages. See Pricing.
Collections or folders
You don't have to group your documents. When you have many, group them so you can search or ask about just some of them. A document goes in a collection or in a folder, not both.
| Collection | Folder | |
|---|---|---|
| Shape | Flat: one list of documents | Nested: folders inside folders |
| Metadata | Yes: add fields such as company and year, and filter by them | No |
| Works with | All three products | PageIndex Agent and Page Search |
| Good for | Searching by what a document is about | Keeping the file structure you already have |