v2.6
DocAI ​
Collection of DocAI functions. Needs the configuration of several underlying services (see Tenant settings). Where documents are mentioned, these can be either PDF documents or images (JPG, PNG, etc). Always use base64 encoded documents/images with data URLs.
Methods ​
ask ​
javascript
ask(document: string, prompt: string, options?: {
/**
* The LLM model to use for this DocAI method. The LLM should be up and running in your environment.
*/
model?: string,
/**
* The maximum amount of tokens to generate. Defaults to 1000.
*/
maxCompletionTokens?: number,
/**
* The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both.
*/
temperature?: number,
/**
* An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both.
*/
top_p?: number,
/**
* The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages.
*/
pages?: string
/**
* Whether to perform pre-OCR on the document. If true, OCR is done first, and the result is fed into the LLM. If false, the LLM will be used against the document itself (usually much faster). Not valid for text-based documents like PDF's.
*/
usePreOcr?: boolean
})Method to ask for information from a document/image. Images are OCRed before querying. This method uses the underlying LLM to answer the question based on the content of the document. It cannot be used to describe images, it only answers based on the text in the document.
Parameters
document string
A base64 document or image that you want to query for information.
prompt string
The prompt/question you want to ask about the document.
options { /* * The LLM model to use for this DocAI method. The LLM should be up and running in your environment. / model?: string, /* * The maximum amount of tokens to generate. Defaults to 1000. / maxCompletionTokens?: number, /* * The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both. / temperature?: number, /* * An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both. / top_p?: number, /* * The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages. / pages?: string /* * Whether to perform pre-OCR on the document. If true, OCR is done first, and the result is fed into the LLM. If false, the LLM will be used against the document itself (usually much faster). Not valid for text-based documents like PDF's. / usePreOcr?: boolean }
(optional) The options for this DocAI method.
Return type
{ result: any, model: string, usage: { promptTokens: number, completionTokens: number, totalTokens: number, timing: any }, error?: string }
Examples
javascript
// Ask a question about a PDF document.
DocAI.ask('data:application/pdf;base64,<base64encodedPDFDocument>',
"Did the valuation reveal any special features?"
);
// Ask a question about an image document, without OCR.
DocAI.ask('data:image/png;base64,<base64encodedImageDocument>', 'What is in this image?', { usePreOcr: false });chat ​
javascript
chat(prompt: string, options?: {
/**
* The LLM model to use for this DocAI method. The LLM should be up and running in your environment.
*/
model?: string,
/**
* The maximum amount of tokens to generate. Defaults to 1000.
*/
maxCompletionTokens?: number,
/**
* The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both.
*/
temperature?: number,
/**
* An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both.
*/
top_p?: number,
})Method to ask the underlying LLM for general information, not related to any document. This returns the response from the LLM along with model and usage information. The result can be either a text or an object.
Parameters
prompt string
The prompt/question you want to ask.
options { /* * The LLM model to use for this DocAI method. The LLM should be up and running in your environment. / model?: string, /* * The maximum amount of tokens to generate. Defaults to 1000. / maxCompletionTokens?: number, /* * The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both. / temperature?: number, /* * An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both. / top_p?: number, }
(optional) The options for the DocAI service. If not specified, the default options will be used.
Return type
{ result: any, model: string, usage: { promptTokens: number, completionTokens: number, totalTokens: number, timing: any }, error?: string }
Examples
javascript
// Ask a question.
DocAI.chat("What to look for in a valuation report?");classify ​
javascript
classify(document: string, possibleTypes: string[], options?: {
/**
* The LLM model to use for this DocAI method. The LLM should be up and running in your environment.
*/
model?: string,
/**
* The maximum amount of tokens to generate. Defaults to 1000.
*/
maxCompletionTokens?: number,
/**
* The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both.
*/
temperature?: number,
/**
* An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both.
*/
top_p?: number,
/**
* The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages.
*/
pages?: string
/**
* Whether to perform pre-OCR on the document. If true, OCR is done first, and the result is fed into the LLM. If false, the LLM will be used against the document itself (usually much faster). Not valid for text-based documents like PDF's.
*/
usePreOcr?: boolean
})Method to classify a document/image.
Parameters
document string
A base64 document or image you want to classify.
possibleTypes string[]
The possible types you expect for the document.
options { /* * The LLM model to use for this DocAI method. The LLM should be up and running in your environment. / model?: string, /* * The maximum amount of tokens to generate. Defaults to 1000. / maxCompletionTokens?: number, /* * The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both. / temperature?: number, /* * An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both. / top_p?: number, /* * The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages. / pages?: string /* * Whether to perform pre-OCR on the document. If true, OCR is done first, and the result is fed into the LLM. If false, the LLM will be used against the document itself (usually much faster). Not valid for text-based documents like PDF's. / usePreOcr?: boolean }
(optional) The options for this DocAI method.
Return type
{ result: { type: string, confidence: number, reasoning: string }, model: string, usage: { promptTokens: number, completionTokens: number, totalTokens: number, timing: any }, error?: string }
Examples
javascript
// Classify a PDF document.
DocAI.classify('data:application/pdf;base64,<base64encodedPDFDocument>', [
"appraisal report",
"passport",
"salary slip"
]);extract ​
javascript
extract(document: string, exampleFormat: { [key: string]: any; }, options?: {
/**
* The LLM model to use for this DocAI method. The LLM should be up and running in your environment.
*/
model?: string,
/**
* The maximum amount of tokens to generate. Defaults to 1000.
*/
maxCompletionTokens?: number,
/**
* The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both.
*/
temperature?: number,
/**
* An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both.
*/
top_p?: number,
/**
* The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages.
*/
pages?: string
/**
* Whether to perform pre-OCR on the document. If true, OCR is done first, and the result is fed into the LLM. If false, the LLM will be used against the document itself (usually much faster). Not valid for text-based documents like PDF's.
*/
usePreOcr?: boolean
})Method to extract information from a document/image.
Parameters
document string
A base64 document or image from which you want to extract information.
exampleFormat { [key: string]: any; }
The format you want the result to have.
options { /* * The LLM model to use for this DocAI method. The LLM should be up and running in your environment. / model?: string, /* * The maximum amount of tokens to generate. Defaults to 1000. / maxCompletionTokens?: number, /* * The temperature to use for the LLM. Defaults to 0.2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. It is recommended to alter either this or top_p but not both. / temperature?: number, /* * An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1. It is recommended to alter either this or temperature but not both. / top_p?: number, /* * The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages. / pages?: string /* * Whether to perform pre-OCR on the document. If true, OCR is done first, and the result is fed into the LLM. If false, the LLM will be used against the document itself (usually much faster). Not valid for text-based documents like PDF's. / usePreOcr?: boolean }
(optional) The options for this DocAI method.
Return type
{ result: { [key: string]: any; }, model: string, usage: { promptTokens: number, completionTokens: number, totalTokens: number, timing: any }, error?: string }
Examples
javascript
// Extract information of a PDF document.
DocAI.extract('data:application/pdf;base64,<base64encodedPDFDocument>', {
address: {
street: "Dorpsstraat",
number: "123",
postcode: "1234 BB",
city: "The Hague"
}
});extractMarkdown ​
javascript
extractMarkdown(document: string, options?: {
/**
* The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages.
*/
pages?: string
/**
* Whether to use Azure's built-in markdown generation. Defaults to false.
*/
useAzureMarkdown?: boolean
})Converts a document to markdown. Returns a string containing the document's markdown. The document can be a PDF, image or other document type. Images are OCRed before conversion.
Parameters
document string
A base64 encoded document or image that you want to convert to markdown.
options { /* * The pages to extract. Can be a single page number (e.g. '1'), a range of pages (e.g. '1-3'). Defaults is to handle all pages. / pages?: string /* * Whether to use Azure's built-in markdown generation. Defaults to false. / useAzureMarkdown?: boolean }
(optional) The options for markdown extraction.
Return type
{ result: string, error?: string, usage: { timing: any } }
Examples
javascript
// Convert a document or image to markdown.
DocAI.extractMarkdown('data:application/pdf;base64,<base64encodedDocument>');extractStructuredData ​
javascript
extractStructuredData(document: string, model: 'prebuilt-idDocument' | 'prebuilt-invoice' | 'prebuilt-receipt' | 'prebuilt-contract', options?: {
/**
* Confidence threshold for extracted data. Defaults to 0.5. If the confidence of a field is below this threshold, it will be omitted from the result.
*/
confidenceThreshold?: number
})Method to extract structured data from a document/image using a pretrained model. The following models are supported: id-documents, invoices, receipts, and contracts.
Parameters
document string
A base64 document or image from which you want to extract structured data.
model 'prebuilt-idDocument' | 'prebuilt-invoice' | 'prebuilt-receipt' | 'prebuilt-contract'
The model defining the structured data to extract from the document.
options { /* * Confidence threshold for extracted data. Defaults to 0.5. If the confidence of a field is below this threshold, it will be omitted from the result. / confidenceThreshold?: number }
(optional) The options for structured data extraction.
Return type
{ result: { [key: string]: any; }, error?: string, usage: { timing: any } }
Examples
javascript
// Extract structured data from a passport:
DocAI.extractStructuredData('data:image/png;base64,<base64encodedPassportImage>', 'prebuilt-idDocument');