Using Google Cloud Function to save JSON payload from Google Storage Event to Google Bigquery
As part of this article we will learn below: Create the cloud storage bucket Use sample hierarchical JSON payload to auto create bigquery table Create google cloud function to listen to the object creation event on storage bucket and Fetch the JSON payload from the file and load in to Bigquery *We will be using Python for this activity. 1. Create the cloud storage bucket This is the simple task, you can either create the bucket using the console or gsutil command. For this exercise, we will use console. We will ignore some parameters like regional, bucket type, as we will not be using this bucket for large storage. Go to cloud storage -> Click on Create Bucket Give some unique name, you can use your Google Project ID+some name and click on create bucket. 2. Use sample hierarchical JSON payload to auto create bigquery table One of the quick way to create the Bigquery table to save the JSON payload with array is to first create the sample JSON payload and use it to create the bigq...
