Tool for uploading files to Blackfynn

Study purpose: The problem many are encountering with uploading to and organizing and renaming files on Blackfynn according to the BIDS-like conventions required by SPARC is that there is a mismatch between how labs organize and name their files locally and the organization of folders and files in datasets on the Blackfynn site.

A solution that some have used is to reorganize and rename things locally. A lot of effort has gone into allowing entire folder/file hierarchies to be uploaded via browser drag and drop. This seems to assume that the structure and naming conventions have been adopted locally.

Data collection: We created a spreadsheet that defines the structure we want on the Blackfynn site and then we list the files we want to be uploaded to that structure. This spreadsheet becomes the input to our Python program. It is easier to understand this if you unzip the zip file and look at some of the example spreadsheets.

Primary conclusion: The upload_bfynn program reads the spreadsheet file, creates folders on the Blackfynn site as required, uploads the files, and then renames them to conform to the BIDS-like naming conventions. Users will not have to make any changes to their local folders and file names.

Curator's Notes

Experimental Design: This is utility software tool.

Completeness: Complete.

Subjects & Samples: Not applicable. This is a utility tool.

Primary vs derivative data: Not applicable.

Code Availability: Under a code folder you will find the subfolder, one for Windows, one for Linux upload. The zip file contains a Python command-line utility written for uploads of files to the Blackfynn site. Later for a windows upload a windows wrapper was added around the command line code. It uses the Blackfynn Python API.

1. create a spreadsheet that creates a template for folders and names that meet the BIDS-like DAT-Core and MAP-Core standards, then

2. list the sources of the files that will be uploaded to these folders, then

3. run the Python program upload_bfynn.py and it will do the work for you.

Important notes: This code has to be updated before use. All SPARC datasets were migrated from Blackfynn.io to Pennsievie.io