How to use the Unified File Storage API

December 25, 2023

The Unified File Storage API allows your application to fetch your customer's files and folders from their favourite cloud storage provider, much like a file manager. These cloud storage providers include Google Drive, Microsoft OneDrive, Dropbox, and more - view all of them here.

This guide will walk you through the File Storage API and how to perform common tasks with it.

Understanding the StorageFile object

The object that is returned from the File Storage API can represent either a file or a folder. This is specified under the type field. Folders do not have a size nor a download_url.

type StorageFile = {
    id?: string;
    created_at?: string; // ISO date
    updated_at?: string; // ISO date
    name?: string;
    description?: string;
    parent_id?: string;
    user_id?: string;
    size?: number;
    type?: 'FILE' | 'FOLDER';
    mime_type?: string;
    permissions?: {
        user_id?: string;
        group_id?: string;
        roles?: any;
    }[];
    download_url?: string;
}

API reference: StorageFile data model

Get a list of files and folders

Most storage providers use a hierarchical file system to represent their contents as a tree. Imagine each node in the tree as either a file or a folder - if it's a folder, that node will have additional branches leading to other files and folders.

When calling the File Storage API List endpoint, we will return the contents (i.e. children) of a node. For example, calling:

GET /storage/{connection_id}/file

Will return the contents under the root node. Notice that no parameters are passed in this API call.

Include parent_id when calling the List endpoint to retrieve the contents of the node (i.e. folder) matching that ID, for example:

GET /storage/{connection_id}/file&parent_id=abc123

Will return the contents of the folder with ID abc123.

API reference: List files and folders

Traverse the file system to get all files

To traverse the file system and retrieve the contents of every subfolder, create a function to recursively call the List endpoint and pass in parent_id for each folder that is found. In other words, use tree recursion to retrieve all the contents of a storage provider's files:

import { UnifiedTo } from '@unified-api/typescript-sdk';

const sdk = new UnifiedTo({
    security: {
        jwt: '<YOUR_API_KEY_HERE>',
    },
});

async function getAllFiles(connectionId: string, parentId?: string): Promise<StorageFile[]> {
    const result = await sdk.storage.listStorageFiles({
        connectionId,
        parentId,
    });

    if (!result.storageFiles) {
        return [];
    }

    let allFiles: StorageFile[] = [];

    for (const item of result.storageFiles) {
        if (item.type === 'FOLDER') {
            // Recursively get files from subfolders
            const subfolderFiles = await getAllFiles(connectionId, item.id);
            allFiles = allFiles.concat(subfolderFiles);
        } else {
            allFiles.push(item);
        }
    }

    return allFiles;
}

Retrieve the contents of a file

If you want to get the data contents of a file, either use the getFile API endpoint or take a File object from the listFiles API endpoint. Use the download_url field to fetch the contents of that file. How you fetch that data will be dependent on your programming language.

Important: The download_url expires after 1 hour. When you need to retrieve the contents of a file, call the getFile endpoint again to get a new download_url.

For example, to retrieve the contents of a file:

import { UnifiedTo } from '@unified-api/typescript-sdk';

const sdk = new UnifiedTo({
    security: {
        jwt: '<YOUR_API_KEY_HERE>',
    },
});

export async function getFileContents(connectionId: string, id: string) {
    // get file details
    const result = await sdk.storage.getStorageFile({
        id,
        connectionId,
    });

    if (!result.storageFile) {
        throw new Error('file not found');
    }

    if (!result.storageFile.downloadUrl) {
        throw new Error('not a downloadable file');
    }

    // Get the data contents of a file
    const contents = fetch(result.storageFile.downloadUrl, { Accept: result.storageFile.mimeType });

    return contents;
}

References

Blog