Saturday, October 29, 2022

Large File Upload from On-premises into Oracle Cloud Object Storage using REST API-Part 1

Large File Upload from On-premises into Oracle
Cloud Object Storage using REST API

In this tutorial(for users of mac os ,linux) , I will walk through about uploading .CSV file from on-premises to Oracle Cloud Object Storage using REST APIs.

When to use this approach ?

Suppose you have large CSV file or else MySQL backup file and you wanted to export into MySQL database service(MDS) , then it’s easy to lift-shift the backup file into Object Storage and from object storage to MySQL database service.

Next blog, we will explore how to import CSV file from Object Storage into MySQL Database Service(MDS).

In case you wanted to know what is object storage then below web link will have detailed info:-

What is Object Storage ?

more info:-

Let’s get started...
High level work flow as below:-

  1. Make sure cURL is install in your On-P machine.

  2. Make sure bucket is created in the object storage of Oracle Cloud

  3. Generate Authentication Token

  4. Generate Pre-Authenticated Requests in side the bucket of Oracle


  5. Run the cURL- PUT command from the On-Premises.

The Oracle Cloud Infrastructure Object Storage service is an internet-

scale, high-performance storage platform that offers reliable and cost-

efficient data durability. The Object Storage service can store an unlimited

amount of unstructured data of any content type, including analytic data

and rich content, like images and videos.

Verify cURL is install or not run curl help , in the command prompt or else install it.

Create bucket in the Oracle Cloud

Bucket is the location where CSV file will be exported from On-Premises. It is a container for storing objects in a compartment within an Object Storage namespace.
You may use for storing data, images, logs, and video etc.

Each object inside bucket is encrypted with its own data encryption key. Data encryption keys are always encrypted with a master encryption key that is assigned to the bucket.
Encryption is enabled by default and cannot be turned off. By default, Oracle manages the master encryption key.

Generate Authentication Token

By going to :- user settingsAuth tokensGenerate Token

Copy the Auth Token and save into text file and looks like below:-


Create a pre-Authenticate Request on Object Storage

Pre-authenticated requests provide a way to let you to access a bucket or an object without having their own credentials.
you continue to have access to the bucket or object as long as the creator of the request has permissions to access those resources.

Go to > storage BucketsCreate BucketPre-Authenticated Requests

Copy the URL as below it will look like

https://objectstorage.ap-mumbai- KNUTviigbc0QlLcQ2IvJXqE5hrZQPC2hsoxkxt/n/idazzjlcjqzj/b/bucket-csv- file-upload/o/

Run the CURL command from On-Premises

cURL is one of the tools used to access the REST interface. cURL is an open source tool used for transferring data which supports various protocols including HTTP and HTTPS.

curl -v -X PUT -u '<user>:<token string from step 2>' --upload-file <local file location> <Pre-authenticated Request URL from step 3>

For example:#1. >> create Pre Authentication Request for Objects

curl -v -X PUT -u'{8CAW2Gr'
--upload-file /Users/cjkumar/Documents/Mac-key/

For example:#2. >> create Pre Authentication Request for Bucket

curl -v -X PUT -u'{8CAW2Gr' --upload-file /Users/cjkumar/Downloads/employee.csv https://objectstorage.ap-mumbai- KNUTviigbc0QlLcQ2IvJXqE5hrZQPC2hsoxkxt/n/idazzjlcjqzj/b/bucket-csv-file-upload/o/

Once done, remember to delete the pre-authenticated Request as it is no longer


Above screenshot show that , we have successfully uploaded CSV file from On-Premises to Oracle Cloud.

Next Blog , I will walk through how to import CSV file from Object storage into MySQL Database Service


Object storage is very easy way to store the data could be structured/unstructured from on-premises to oracle cloud and use with cloud native applications.
REST API uses cURL which simplifying the task of lift-shift from different sources into Oracle Cloud object storage.
REST API is used to manage containers and objects in the Oracle Storage Cloud Service instance.
Anyone can access the REST API from any application or programming platform that understands the Hypertext Transfer Protocol (HTTP) and has Internet connectivity.