Sample code to upload files into Dropbox using NodeJS (both single upload and multi-part upload)

Overview

New web applications support functionalities to store files/images/videos/etc and there are many options in executing this. One could keep it inside the database, store within the local file storage, S3 buckets, and the list goes on. Best practice would be to store it in a separate entity so that scaling is possible for both the web server and file server.

In this article, I will go through the option of storing it in Dropbox. If you need tutorial on how to create and handle file uploads to a NodeJS, you can read my previous article File Upload in NodeJS with ExpressJS.

Steps

1. Create Dropbox App and generate App configration— Go to https://www.dropbox.com/developers/apps and create your app.

Click on the “Create app” button

There isn’t much options to choose.

Complete your app details

Set the Permissions settings before generating the token as it remembers the settings.

Next, go to the Permissions tab

Copy your token from the Settings tab. You can also re-generate for a new token here if it has been compromise. Do not check-in these information in your configuration of your code. Use the proper method to store and retrieve them.

Go back to Settings tab

At the Settings tab

2. Create NodeJS app

2.1 Let’s test with basic test code to see if it works using the token generated in step 1. The code below is a simple post to Dropbox API to read the number of files inside. The detailed documentation can be found here — https://www.dropbox.com/developers/documentation/http/documentation#files-upload_session-start

Results from running the code above

Seems like with the token, I am able to successfully query the number of files inside.

2.2 Next let’s try to write code that uploads a file that is less than 150MB to Dropbox. For this we will use the API — https://www.dropbox.com/developers/documentation/http/documentation#files-upload

Following the API documentation

Success!

2.3 Uploading large files to Dropbox by parts…

If your file is larger than 150MB, Dropbox provide a different method to upload your file by breaking it up into parts of 150MB. However, your file should still be lesser than 350GB.

What if my files are larger than 150MB?

Dropbox provides another set of APIs(upload_session) to allow the file to be uploaded by parts.

1. Create session (/upload_session/start)
2. Append file content for session (/upload_session/append)
3. Close session (/upload_session/finish)

There are a lot more room for errors. Large files stored within the server could potentially overwhelm the server.

The code can be divided into the following parts:

1. Create new upload session
2. Open local file for content
3. For each stream data, post it to server via append
4. On end of file stream, close the session

One of the pitfall in this development code was the size of the offset. Since the offset is large, we will need BigInt and the standard JSON is unable to stringify BigInt. Here, we make use of JSON-BigInt library to solve this issue.

Conclusion

Performing upload of images to Dropbox is fairly simple provide the size is small (<150MB). Dropbox API is relatively simple and clear cut.

However, I would not recommend this method for large images. The uploading is complex with many point of failures. The team will need to accurately handle all possible cases of failure correctly.