U DAfa @sddlZddlZddlmZddlmZddlmZmZm Z m Z m Z m Z ddl ZddlZddlmZmZmZmZmZddlmZmZddlmZddlmZdd lmZm Z e!e"Z#ed d *Z$dd l%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+W5QRXeGdddZ,dS)N) defaultdict)Path)AnyDictListLiteralOptionalUnion)Document componentdefault_from_dictdefault_to_dictlogging)get_bytestream_from_sourcenormalize_metadata) ByteStream) LazyImport)Secretdeserialize_secrets_inplacez4Run 'pip install "azure-ai-formrecognizer>=3.2.0b2"')message) AnalyzeResultDocumentAnalysisClient DocumentLineDocumentParagraph)AzureKeyCredentialc @seZdZdZedddddddfeeeeeee de e d d d Z e jeeeed d.eeeeefe eeeefdddZeeefdddZeeeefddddZde eeefeedddZde eeefeedddZde eeefedddZd/de eeefe edd d!Zded"d#d$Zeed%ed&d'd(Z d0e!j"ed+d,d-Z#d S)1AzureOCRDocumentConvertera Convert files to documents using Azure's Document Intelligence service. Supported file formats are: PDF, JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML. In order to be able to use this component, you need an active Azure account and a Document Intelligence or Cognitive Services resource. Follow the steps described in the [Azure documentation] (https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/quickstarts/get-started-sdks-rest-api) to set up your resource. Usage example: ```python from haystack.components.converters import AzureOCRDocumentConverter from haystack.utils import Secret converter = AzureOCRDocumentConverter(endpoint="", api_key=Secret.from_token("")) results = converter.run(sources=["path/to/doc_with_images.pdf"], meta={"date_added": datetime.now().isoformat()}) documents = results["documents"] print(documents[0].content) # 'This is a text from the PDF file.' ``` ZAZURE_AI_API_KEYz prebuilt-readTnatural皙?)r single_column)endpointapi_keymodel_idpreceding_context_lenfollowing_context_lenmerge_multiple_column_headers page_layout threshold_yc Csptt|t|pdd|_||_||_||_||_ ||_ ||_ ||_ ||_ |j dkrl|j dkrld|_ dS)a Create an AzureOCRDocumentConverter component. :param endpoint: The endpoint of your Azure resource. :param api_key: The key of your Azure resource. :param model_id: The model ID of the model you want to use. Please refer to [Azure documentation] (https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/choose-model-feature) for a list of available models. Default: `"prebuilt-read"`. :param preceding_context_len: Number of lines before a table to extract as preceding context (will be returned as part of metadata). :param following_context_len: Number of lines after a table to extract as subsequent context ( will be returned as part of metadata). :param merge_multiple_column_headers: Some tables contain more than one row as a column header (i.e., column description). This parameter lets you choose, whether to merge multiple column header rows to a single row. :param page_layout: The type reading order to follow. If "natural" is chosen then the natural reading order determined by Azure will be used. If "single_column" is chosen then all lines with the same height on the page will be grouped together based on a threshold determined by `threshold_y`. :param threshold_y: The threshold to determine if two recognized elements in a PDF should be grouped into a single line. This is especially relevant for section headers or numbers which may be spacially separated on the horizontal axis from the remaining text. The threshold is specified in units of inches. This is only relevant if "single_column" is chosen for `page_layout`. )r Z credentialrNr) azure_importcheckrrZ resolve_valuedocument_analysis_clientr r"r!r#r$r%r&r') selfr r!r"r#r$r%r&r'r-H/tmp/pip-unpacked-wheel-z752163x/haystack/components/converters/azure.py__init__4s%z"AzureOCRDocumentConverter.__init__ documentsZraw_azure_responseN)sourcesmetac Csg}g}t|t|d}t||D]\}}zt|d}Wn:tk rr} ztjd|| dWYq"W5d} ~ XYnX|jj|j |j d} | } | | |j|} |j| | d} || q"||dS) az Convert a list of files to Documents using Azure's Document Intelligence service. :param sources: List of file paths or ByteStream objects. :param meta: Optional metadata to attach to the Documents. This value can be either a list of dictionaries or a single dictionary. If it's a single dictionary, its content is added to the metadata of all produced Documents. If it's a list, the length of the list must match the number of sources, because the two lists will be zipped. If `sources` contains ByteStream objects, their `meta` will be added to the output Documents. :returns: A dictionary with the following keys: - `documents`: List of created Documents - `raw_azure_response`: List of raw Azure responses used to create the Documents )r3Z sources_count)sourcez4Could not read {source}. Skipping it. Error: {error})r4errorN)r"documentresultr3r0)rlenzipr Exceptionloggerwarningr+Zbegin_analyze_documentr"datar8appendto_dictr3_convert_tables_and_textextend)r,r2r3r1Z azure_outputZ meta_listr4metadataZ bytestreameZpollerr8Zmerged_metadatadocsr-r-r.runis&  zAzureOCRDocumentConverter.run)returnc Cs.t||j|j|j|j|j|j|j|j d S)z{ Serializes the component to a dictionary. :returns: Dictionary with serialized data. )r!r r"r#r$r%r&r') r r!r@r r"r#r$r%r&r')r,r-r-r.r@sz!AzureOCRDocumentConverter.to_dict)r>rGcCst|ddgdt||S)z Deserializes the component from a dictionary. :param data: The dictionary to deserialize from. :returns: The deserialized component. Zinit_parametersr!)keys)rr )clsr>r-r-r. from_dicts z#AzureOCRDocumentConverter.from_dictr)r8r3rGcCsX|j||d}|jdkr(|j||d}n"t|jts8t|j|||jd}||f}|S)a Converts the tables and text extracted by Azure's Document Intelligence service into Haystack Documents. :param result: The AnalyzeResult object returned by the `begin_analyze_document` method. Docs on Analyze result can be found [here](https://azuresdkdocs.blob.core.windows.net/$web/python/azure-ai-formrecognizer/3.3.0/azure.ai.formrecognizer.html?highlight=read#azure.ai.formrecognizer.AnalyzeResult). :param meta: Optional dictionary with metadata that shall be attached to all resulting documents. Can be any custom keys and values. :returns: List of Documents containing the tables and text extracted from the AnalyzeResult object. r7r)r8r3r')_convert_tablesr&_convert_to_natural_text isinstancer'floatAssertionError_convert_to_single_column_text)r,r8r3tablestextrEr-r-r.rAs   z2AzureOCRDocumentConverter._convert_tables_and_textc s g}|js|S|jD]fddtjD}t}d}d}tjD]\}} | jdd| _| jdd| _|dkr| jj kr| j}d}| dqJ| jr| jnd} t| D]} | j r| j nd} t| D]v} |j r*| j dkr*| j|kr*|d| j| d | j7<|| j|q| j|| j| || j| <qqqJt|d d D] }||=qZjrtfd d |jD}nd}jdj|r|jrfdd|jD}ng}d ||j dd |}|}jrtjdkr|}n&jr*tfdd |jD}nd}jdj|rb|jrbfdd|jD}ng}d |d|j}t|}t |t!r||d<||d<n ||d}jrĈjdj"|d<t#j$|d|ddd}|%|}||}t&'|()}|*t+|||dq|S)av Converts the tables extracted by Azure's Document Intelligence service into Haystack Documents. :param result: The AnalyzeResult Azure object :param meta: Optional dictionary with metadata that shall be attached to all resulting documents. :returns: List of Documents containing the tables extracted from the AnalyzeResult object. csg|]}dgjqS)r() column_count).0_tabler-r. sz=AzureOCRDocumentConverter._convert_tables..r(rz :selected:z :unselected:Z columnHeader T)reversec3s$|]}|jjdjkr|VqdS)rN page_numberbounding_regionsrTpagerVr-r. sz.Ncs"g|]}|jdjkr|jqSrspansoffsetcontentrTline)table_start_offsetr-r.rXsc3s$|]}|jjdjkr|VqdS)Nr\r_rVr-r.rascs"g|]}|jdjkr|jqSrbrcrg)table_end_offsetr-r.rXspreceding_contextfollowing_context)rlrmr`)columnsr>)idZ dataframer3),rQrangeZ row_countset enumeratecellsrfreplace column_spanrSpoprow_spanr%kindZ row_indexZ column_indexaddsortedr^nextpagesrdrelinesjoinr#stripr9lengthr$copydeepcopyrMdictr]pd DataFrame_hash_dataframehashlibsha256encode hexdigestr?r )r,r8r3Zconverted_tablesZ table_listZadditional_column_header_rowscaptionZ row_idx_startidxcellrucrwrZrow_idxZtable_beginning_pageZpreceding_linesrlZtable_end_pageZfollowing_linesrmZ table_metaZtable_dfZ pd_hashesr>Zdoc_idr-)rWrkrir.rKs     "$               z)AzureOCRDocumentConverter._convert_tablescCs|j|d}g}|jrtt}|jD]p}|jr@dd|jD}n|rTt|dnd}|g}||d} |j| |drzq$||d|jd7<q$t |} t d| dD]} | | d } | | qn t d d |} t| |r|nid S) a( This converts the `AnalyzeResult` object into a single document. We add " " separators between to differentiate between the text on separate pages. This is the expected format for the PreProcessor. :param result: The AnalyzeResult object returned by the `begin_analyze_document` method. Docs on Analyze result can be found [here](https://azuresdkdocs.blob.core.windows.net/$web/python/azure-ai-formrecognizer/3.3.0/azure.ai.formrecognizer.html?highlight=read#azure.ai.formrecognizer.AnalyzeResult). :param meta: Optional dictionary with metadata that shall be attached to all resulting documents. Can be any custom keys and values. :returns: A single Document containing all the text extracted from the AnalyzeResult object. r8cSsg|] }|jqSr-)r])rTbr-r-r.rXIszFAzureOCRDocumentConverter._convert_to_natural_text..rjrYrline_or_paragraphrZr(z7No text paragraphs were detected by the OCR conversion. rfr3)_collect_table_spansZ paragraphsrstrr^rzrH_check_if_in_tablerfmaxrpgetr?r<r=r~r )r,r8r3table_spans_by_pagetextsZparagraphs_to_pagesZ paragraphZ page_numbersZcurrent_last_page_numbertables_on_pageZmax_page_numberpage_idx page_textall_textr-r-r.rL4s(      z2AzureOCRDocumentConverter._convert_to_natural_text)r8r3r'rGcsj|d}tt}t|jD]\}}|jr2|jngtddDrttD]}|j \} } } } || ||gt|dtD]D} | j \} } } } t | d| d|k} | r|| || gqqTqt dj|dttD]}|| ||gqqi}|D]8}t}|||ddtt|D||<qi}t|jD]X\}}g}|jrz|jng||gD]"}fd d|D}| |q|||<q`i}t|jD]<\}} g}||D]}| t|d d d q|||<qi}t|jD]&\}} t||d d d }|||<qg}t|jD]p\}}||jd}||D]D}tfdd|Drqh|ddd|D7}|d7}qh| |qJd|}t||r|nidS)a This converts the `AnalyzeResult` object into a single Haystack Document. We add " " separators between to differentiate between the text on separate pages. This is the expected format for the PreProcessor. :param result: The AnalyzeResult object returned by the `begin_analyze_document` method. Docs on Analyze result can be found [here](https://azuresdkdocs.blob.core.windows.net/$web/python/azure-ai-formrecognizer/3.3.0/azure.ai.formrecognizer.html?highlight=read#azure.ai.formrecognizer.AnalyzeResult). :param meta: Optional dictionary with metadata that shall be attached to all resulting documents. Can be any custom keys and values. :param threshold_y: height threshold in inches for PDF and pixels for images :returns: A single Document containing all the text extracted from the AnalyzeResult object. rcss|]}|jdk VqdSNpolygonrgr-r-r.raxszKAzureOCRDocumentConverter._convert_to_single_column_text..rYzPolygon information for lines on page {page_idx} is not available so it is not possible to enforce a single column page layout.)rcSsg|] }t|qSr-)list)rTar-r-r.rXszLAzureOCRDocumentConverter._convert_to_single_column_text..csg|] }|qSr-r-)rTZline_idx)r}r-r.rXscSs|jddS)Nrrxr-r-r.zJAzureOCRDocumentConverter._convert_to_single_column_text..)keycSs|djddS)NrrYrrr-r-r.rrr(c3s|]}j|dVqdS)rN)rrg)r,rr-r.ras css|] }|jVqdSr)rfrgr-r-r.rasrZrr)rrrrrr|r}allrpr9rr?absr<infoformatnxZGraphZadd_edges_fromZconnected_componentsrrzr]anyr~r )r,r8r3r'rZ pairs_by_pagerr`iZleft_upirUjZleft_upjZclose_on_y_axisZmerged_pairs_by_pagegraphZmerged_lines_by_pageZrowsZ row_of_linesZ lines_in_rowZx_sorted_lines_by_pageZ sorted_rowsZy_sorted_lines_by_pagerrrr-)r}r,rr.rP`sn          z8AzureOCRDocumentConverter._convert_to_single_column_text)r8rGcCsJtt}|jr|jng}|D](}|js(q||jdj|jdq|S)a Collect the spans of all tables by page number. :param result: The AnalyzeResult object returned by the `begin_analyze_document` method. :returns: A dictionary with the page number as key and a list of table spans as value. r)rrrQr^r]r?rd)r,r8rrQrWr-r-r.rsz.AzureOCRDocumentConverter._collect_table_spans)rr)rrrGcCsDd}|D]6}|j|jdjkr2|j|jkrnqd}q@q|S)aF Check if a line or paragraph is part of a table. :param tables_on_page: A dictionary with the page number as key and a list of table spans as value. :param line_or_paragraph: The line or paragraph to check. :returns: True if the line or paragraph is part of a table, False otherwise. FrT)rerdr)r,rrZin_tablerWr-r-r.rs *z,AzureOCRDocumentConverter._check_if_in_table)dfrGc Csnt}t|}td||}tjj|dd}|dd|}|D]$} t| d|d} | | q@| S)a_ Returns a hash of the DataFrame content. The hash is based on the content of the DataFrame. :param df: The DataFrame to hash. :param desired_samples: The desired number of samples to hash. :param hash_length: The length of the hash for each sample. :returns: A hash of the DataFrame content. rYT)indexNzutf-8) rmd5r9rrutilZhash_pandas_objectrrupdater) r,rZdesired_samplesZ hash_lengthZhasherZ total_rowsZ sample_ratehashesZsampled_hashesZ hash_valueZ partial_hashr-r-r.rs  z)AzureOCRDocumentConverter._hash_dataframe)N)r)rr)$__name__ __module__ __qualname____doc__rZ from_env_varrintboolrrrNr/r Z output_typesrr rr rrrrFr@ classmethodrJrArKrLrPrrrrrrr-r-r-r.rsN 52( ""n- Z r)-rr collectionsrpathlibrtypingrrrrrr ZnetworkxrZpandasrZhaystackr r r r rZ$haystack.components.converters.utilsrrZhaystack.dataclassesrZhaystack.lazy_importsrZhaystack.utilsrr getLoggerrr<r)Zazure.ai.formrecognizerrrrrZazure.core.credentialsrrr-r-r-r.s"