How to use the ginza.sudachi_tokenizer.SudachiTokenizer function in ginza
To help you get started, we’ve selected a few ginza examples, based on popular ways it is used in public projects.
Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.
megagonlabs / ginza / ginza / __init__.py View on Github 
def create_tokenizer(cls, nlp=None):
return SudachiTokenizer(nlp)ginza
GiNZA, An Open Source Japanese NLP Library, based on Universal Dependencies
