pliers.filters.TokenRemovalFilter¶
- class pliers.filters.TokenRemovalFilter(tokens=None, language='english')[source]¶
Bases:
TextFilter
Removes tokens (e.g., stopwords, common words, punctuation) from a TextStim.
- Parameters
- transform(stims, validation='strict', *args, **kwargs)¶
Executes the transformation on the passed stim(s).
- Parameters
One or more stimuli to process. Must be one of:
A string giving the path to a file that can be read in as a Stim (e.g., a .txt file, .jpg image, etc.)
A Stim instance of any type.
An iterable of stims, where each element is either a string or a Stim.
validation (str) –
String specifying how validation errors should be handled. Must be one of:
’strict’: Raise an exception on any validation error
’warn’: Issue a warning for all validation errors
’loose’: Silently ignore all validation errors
args – Optional positional arguments to pass onto the internal _transform call.
kwargs – Optional positional arguments to pass onto the internal _transform call.