How to use the pyclowder.datasets.download_metadata function in pyclowder

To help you get started, we’ve selected a few pyclowder examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github terraref / computing-pipeline / scripts / globusmonitor / globus_uploader_service.py View on Github external
dsFileList = fetchDatasetFileList(dsid, sess)
                    # Only send files not already present in dataset by path
                    for queued in filesQueued:
                        alreadyStored = False
                        for storedFile in dsFileList:
                            if queued[0] == storedFile['filepath']:
                                logger.info("   skipping file %s (already uploaded)" % queued[0])
                                alreadyStored = True
                                break
                        if not alreadyStored:
                            fileFormData.append(("file",'{"path":"%s"%s}' % (queued[0], queued[1])))

                    if datasetMD and cleaned_dsmd:
                        # Check for existing metadata from the site user
                        alreadyAttached = False
                        md_existing = download_metadata(None, hierarchy_host, clowder_key, dsid)
                        for mdobj in md_existing:
                            if 'agent' in mdobj and 'user_id' in mdobj['agent']:
                                if mdobj['agent']['user_id'] == "https://terraref.ncsa.illinois.edu/clowder/api/users/%s" % clowder_id:
                                    logger.info("   skipping metadata (already attached)")
                                    alreadyAttached = True
                                    break
                        if not alreadyAttached:
                            md = {
                                "@context": ["https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
                                             {"@vocab": clowder_context}],
                                "content": cleaned_dsmd,
                                "agent": {
                                    "@type": "cat:user",
                                    "user_id": "https://terraref.ncsa.illinois.edu/clowder/api/users/%s" % clowder_id
                                }
                            }