How to use the b2luigi.core.utils.flatten_to_file_paths function in b2luigi

To help you get started, we’ve selected a few b2luigi examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github nils-braun / b2luigi / b2luigi / core / task.py View on Github external
def _transform_input(input_generator, key=None):
        input_list = utils.flatten_to_list_of_dicts(input_generator)
        file_paths = utils.flatten_to_file_paths(input_list)

        if key is not None:
            return file_paths[key]
        else:
            return file_paths
github nils-braun / b2luigi / b2luigi / core / tasks.py View on Github external
def get_input_file_names(self):
        return_dict = collections.defaultdict(list)

        for i in self.input():
            file_names = utils.flatten_to_file_paths(utils.flatten_to_dict(i))

            for key, file_name in file_names.items():
                return_dict[key].append(file_name)

        return {key: value for key, value in return_dict.items()}
github nils-braun / b2luigi / b2luigi / core / tasks.py View on Github external
def get_output_file_names(self):
        return utils.flatten_to_file_paths(
            utils.flatten_to_dict(self.output())
            )
github nils-braun / b2luigi / b2luigi / core / task.py View on Github external
"""
        Analogous to :obj:`get_input_file_names` this function returns
        a an output file defined in out output function with
        the given key.

        In contrast to :obj:`get_input_file_names`, only a single file name
        will be returned (as there can only be a single output file with a given name).

        Args:
            key (:obj:`str`): Return the file path with this given key.

        Return:
            Returns only the file path for this given key.
        """
        target = self._get_output_target(key)
        file_paths = utils.flatten_to_file_paths(target)

        return file_paths