U 5Af. @sRddlmZddlmZddlmZmZmZeedddd Gd d d eZd S) )Dict)add_end_docstrings) GenericTensorPipelinebuild_pipeline_init_argsTF)Z has_tokenizerZsupports_binary_outputa tokenize_kwargs (`dict`, *optional*): Additional dictionary of keyword arguments passed along to the tokenizer. return_tensors (`bool`, *optional*): If `True`, returns a tensor according to the specified framework, otherwise returns a list.csReZdZdZdddZeeefdddZdd Z dd d Z fd dZ Z S)FeatureExtractionPipelinea Feature extraction pipeline uses no model head. This pipeline extracts the hidden states from the base transformer, which can be used as features in downstream tasks. Example: ```python >>> from transformers import pipeline >>> extractor = pipeline(model="google-bert/bert-base-uncased", task="feature-extraction") >>> result = extractor("This is a simple test.", return_tensors=True) >>> result.shape # This is a tensor of shape [1, sequence_length, hidden_dimension] representing the input string. torch.Size([1, 8, 768]) ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This feature extraction pipeline can currently be loaded from [`pipeline`] using the task identifier: `"feature-extraction"`. All models may be used for this pipeline. See a list of all models, including community-contributed models on [huggingface.co/models](https://huggingface.co/models). NcKsN|dkr i}|dk r,d|kr$td||d<|}i}|dk rD||d<|i|fS)N truncationz\truncation parameter defined twice (given as keyword argument as well as in tokenize_kwargs)return_tensors) ValueError)selfr tokenize_kwargsr kwargsZpreprocess_paramsZpostprocess_paramsrM/tmp/pip-unpacked-wheel-zw5xktn0/transformers/pipelines/feature_extraction.py_sanitize_parameters(sz.FeatureExtractionPipeline._sanitize_parameters)returncKs|j|fd|ji|}|S)Nr ) tokenizer framework)r inputsr model_inputsrrr preprocess;sz$FeatureExtractionPipeline.preprocesscCs|jf|}|S)N)model)r r model_outputsrrr_forward?s z"FeatureExtractionPipeline._forwardFcCs@|r |dS|jdkr"|dS|jdkr<|dSdS)Nrpttf)rtolistZnumpy)r rr rrr postprocessCs    z%FeatureExtractionPipeline.postprocesscstj||S)a Extract the features of the input(s). Args: args (`str` or `List[str]`): One or several texts (or one list of texts) to get the features of. Return: A nested list of `float`: The features computed by the model. )super__call__)r argsr __class__rrr!Ls z"FeatureExtractionPipeline.__call__)NNN)F) __name__ __module__ __qualname____doc__rrstrrrrrr! __classcell__rrr#rr s    r N) typingrutilsrbaserrrr rrrrs