Upload and manage documents

Upload a document once, and use it with PageIndex Agent, Page Search and Page Extract. A document is one file, such as a PDF. It is made of pages, numbered from 1, and search results, citations and extractions all point back to them.

Set up your client as in the quick start first. With Page Search or Page Extract, use client.pageindex wherever these examples use client. For HTTP requests, see the API reference.

Upload

PageIndex takes PDFs, Word documents (.docx, .doc) and PowerPoint presentations (.pptx, .ppt). Page Extract takes PDFs only.

How you upload depends on the product you use:

You useUpload withPrice
PageIndex AgentPageIndexClient.submit_document()$1 per 1,000 pages
Page SearchPageSearchClient.upload()$0.10 per 1,000 pages
Page ExtractPageExtractClient.extract(), which uploads the file for youFree: you pay only for the extraction
# wait=True waits until document preprocessing is ready
doc_id = client.submit_document("./report.pdf", wait=True)["doc_id"]

The upload returns the document's ID. Keep it: every other call takes it.

This upload preprocesses the document for PageIndex Agent. If you only use Page Search, upload this way instead:

from pageindex import PageSearchClient

doc_id = PageSearchClient().upload("./report.pdf")["doc_id"]  # waits until the PDF is ready to search

A document uploaded for Page Search or Page Extract still works with PageIndex Agent, and you can build its outline later.

To put the document in a collection or a folder, see Collections or folders.

Document status

A document isn't ready the moment you upload it. It moves through these statuses:

StatusWhat it means
queuedWaiting to start.
processingPageIndex is reading the pages.
completedReady to use.
failedIt couldn't be read.

The upload examples above wait until the document is completed. If it fails, the wait raises an error. The wait also gives up after 30 minutes, though the document goes on processing.

To upload without waiting in Python, leave out wait=True, then call client.wait_document(doc_id) when you need the document.

Build an outline later

A document uploaded for Page Search or Page Extract has no outline for PageIndex Agent to follow. You can build one later, at $1 per 1,000 pages, the same as an upload for PageIndex Agent.

client.build_outline_index(doc_id)  # waits until the outline is built

Manage documents

Check a document

doc = client.get_document(doc_id)
print(doc["name"], doc["status"], doc["pageNum"])

You get the document's name, status, page count, and its folder, collection and metadata if it has them. For what each status means, see Document status.

List documents

page = client.list_documents(limit=20, offset=0)
for doc in page["documents"]:
    print(doc["id"], doc["name"], doc["status"])

print(page["total"])  # how many documents you have

Documents come newest first, 50 at a time unless you set limit.

To list only the documents in one collection or one folder, see Collections with metadata and List documents in a folder.

Read a document

Get the text of some pages, or a page as an image.

# The text of pages 5 to 7
for page in client.get_page_content(doc_id, "5-7"):
    print(page["page_index"], page["markdown"][:200])

# Page 5 as an image, in JPEG bytes
image = client.page_image(doc_id, 5)

Delete a document

client.delete_document(doc_id)

You can't delete a document while it is still queued or processing. Wait for it first.

Storage

Storing documents is free up to 10,000 pages. To store more, turn on usage-based billing in the PageIndex Console: storage past 10,000 pages then costs $0.10 per 1,000 pages a month. Without it, requests are refused once you hold more than 10,000 pages. See Pricing.

Collections or folders

You don't have to group your documents. When you have many, group them so you can search or ask about just some of them. A document goes in a collection or in a folder, not both.

CollectionFolder
ShapeFlat: one list of documentsNested: folders inside folders
MetadataYes: add fields such as company and year, and filter by themNo
Works withAll three productsPageIndex Agent and Page Search
Good forSearching by what a document is aboutKeeping the file structure you already have

See Collections with metadata and File system with folders.