Files

Upload local files to Hex, or import them from a cloud file directory.

Add files to your project

You can add up to 100 files (<2GB each) to any project from the Files sidebar. The most common use case is uploading a CSV as a data source.

tip

Hex downloads files at the time of import, and again during kernel starts. Unused files are still downloaded each time a kernel starts and can lead to longer kernel load times. We recommend deleting unused files to optimize project performance.

Local files

Files from your local machine can be uploaded by dragging and dropping your files into the Drop files here area or clicking the browse files option and selecting files to upload.

External files

Admins can set up external file integrations which allow users to import files from external storage solutions into projects. Under the External file integrations header, click Import on the integration you want to import files from. Then, click on the integration and select the files you want to import into the project.

File directory

Every project has its own directory, /hex, where local imported files are stored. If files from an external source are present, they are stored in subdirectories of the /hex/external-files directory.

caution

Any local files created in a sub directory will not be persisted after the project's kernel has been restarted.

Referencing imported files

Local files are located in the current working directory and can be referenced by name. External files will be located in a subdirectory with the naming pattern external-files/${integration_type}/${integration_name}. The integration name is lower-cased and sanitized to remove any non-alphanumeric characters, with spaces replaced by underscores. For example, the path for an S3 integration named "Hex's Testing Bucket" would be external-files/s3/hexs_testing_bucket. The path for a Google Drive integration named "Hex's Test Drive" would be external-files/googledrive/hexs_test_drive.

You can always copy the filename, including its path, from the three-dot menu next to the file. For CSV files, you can copy the DataFrame creation code, or create a SQL Cell with the CSV as its source.

Download files

You can download any imported files from the file browser.

Export files to external file integrations

Hex watches the external file directories for new files and updates to existing files, which will automatically export or push updates to the external source. For example, if a .csv file is created from a dataframe using dataframe.to_csv('external-files/s3/hexs_testing_bucket/my_new_data.csv'), Hex will write the file my_new_data.csv to the bucket in S3. Similarly, dataframe.to_csv('external-files/googledrive/hexs_test_drive/my_new_data.csv') will write the file my_new_data.csv to the Google Drive folder designated in the integration configuration.

Note that files created in this way will not automatically be imported into the project and will no longer be present once the kernel restarts.

caution

The credentials for your external file integration will need to have permissions to write data in order for Hex to successfully create or update the file.

Files in different run contexts

Local files written in Notebook sessions are persisted between runs of the project. Published apps and scheduled runs execute with different run contexts, and local files written during runs in these contexts will not be persisted. If you want to save a file during a scheduled run or published app session, the data must be written to an external source, such as your database, or as external files.

Frequently asked questions

Render images

You can use a Markdown or Text cell to display images in your project.

You can also render imported image files through the use of IPython.display, using code like the following in a Python cell:

from IPython.display import Image
Image(filename='file.png')

Duplicate a project with imported files

When you duplicate a project, local files in the main /hex directory and all external files (as visible in the Files tab) will be included in the new duplicate. If you have created any local files in subdirectories, their contents will not be duplicated with the new project.

View all files in project

To see what files exist in your current project, check out the Files tab of the left sidebar or use this code in a Python cell:

from os import walk
from pathlib import PurePath
for path, subdirs, files in walk('.'):
    for name in files:
        print(PurePath(path, name))

Delete all local files in project

To programmatically delete all local files in your project's directory, you can loop through the contents of your directory:

import os
for file in os.listdir():
    if os.path.isfile(file):
        os.remove(file)

warning

Deleted files are not recoverable. Be certain you want to delete all files in your project before using this code.