View all openai analysis

How to use the openai.text_utils.TextEncoder function in openai

To help you get started, we’ve selected a few openai examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

Separius / BERT-keras / data / vocab.py View on Github

def __init__(self, encoder_path: str = './openai/model/encoder_bpe_40000.json',
                 bpe_path: str = './openai/model/vocab_40000.bpe') -> None:
        self.encoder = _OpenAITextEncoder(encoder_path, bpe_path)
        super().__init__(len(self.encoder.encoder))