Posts

Journey towards launching: Follow My Church Mobile App - (iOS & Android)

At the start of 2023 I wanted to work on a pet project to implement Cloud Native App.  Following a series of brainstorming sessions, I reached the decision to develop an application specifically designed for churches.  The primary objective is to create a multi-tenant app that enables each individual church to independently manage its own content, while allowing users to access and view the relevant content for their respective churches through a mobile app. I will attempt to guide you through the entire process of implementing this pet project. ***As this project was undertaken as a personal endeavor, the objective was to minimize costs by predominantly utilizing open source technologies and free tier products*** Follow My Church - High Level Components  Phase 1:  Develop   Prototype with App Features for feasibility study Created a prototype which will help visualize the To-Be product and its features.         . .... Too
Read more

Combine or Merge XML documents in Single XML using Boomi & Groovy

Image
In this small post will try to show how you can merge multiple XML Documents in Boomi.  Consider following scenario where you have to merge Document 1 & 2 in a new Document 3 Document 1: <?xml version="1.0" encoding="UTF-8"?> <tns:OutputParameters xmlns:tns="http://sample.com">    <tns:Book>       <tns:Names>          <tns:Name>XZY World</tns:Name>       </tns:Names>       <tns:Authors>          <tns:AuthorID>12354</tns:AuthorID>       </tns:Authors>    </tns:Book> </tns:OutputParameters>   Document 2: <?xml version="1.0" encoding="UTF-8"?> <tns:OutputParameters xmlns:tns="http://sample.com">    <tns:Book>       <tns:Names>          <tns:Name>ABC World</tns:Name>       </tns:Names>       <tns:Authors>          <tns:AuthorID>53171</tns:AuthorID&g
Read more

Quick Guide - Docker/Container/Container Images/Registry

In this post we will be listing and exploring the commands and basic approach to get started build container images with Docker. I am using Google Cloud Shell for this blog but you just installed Docker on your local machine.  Lets go through each keyword in brief: Container Images:  Container image is the package of the actual code and all its dependencies. This makes it easy to be executed on any environments/OS etc.  Example, if you have a nodeJs file say app.js. In order to run the app.js, you will need node version to be installed on the machine. Similarly, in case of a container Image, it will first have a node installation -> your code i.e. app.js in a single package.  Container: Container Images become container when they are running on the platform example, Docker/Kubernetes etc.  Registry: Once you build the image on the local machine, you can check in the image in to Container Registry. This allow it to be pulled from multiple platforms and make it sharable.  Ref in case
Read more

Getting started with Data Engineering - Google Cloud Notes

Data engineering at high level is to design, build, monitor and secure the data processing pipelines.  Data pipelines mainly perform ETL (Extract, Transform, Load) jobs, which transform the source data (structure/un-structure) to a data warehouse in a meaningful way, which can than be used for analytics or machine learning models.  Data processing can be in form of batch or streaming. On GCP, we can use Dataproc (Apache Hadoop, Spark etc.) for Batch processing and Dataflow (Apache Beam programing) for Batch & Streaming.  Batch Pipeline: Processing the data in the form of batches. Examples: A nightly Job to extract sales transactions for analysis.  Streaming  Pipeline: Processing continues stream of data. Examples: IoT events, payment processing events, logs. Streaming processing is used when you need a near real-time outcome or decision making.  To get started on Data engineering, would recommend you to go through below courses or labs.  Modernizing Data Lakes and Data Warehouses w
Read more

Introduction to Google Apigee X

Image
  Google Apigee X is the API management platform which allow to develop and manager experience API's.  This APIs can be made available to multiple clients with different security policies.  Example, a Mobile Developer might need a JWT access token validation policy vs a traditional system might need same API with HTTP Basic policy.  As an API management platform, it allow several security policies like SQL injections, JSON Threat Protection etc.  Google Apigee X is full managed API management platform. It consist of a management plane and runtime plane. You can also implement Hybrid model i.e. Cloud & On-Premise runtime instances.  You can refer, below link for high level architecture: https://cloud.google.com/apigee/docs/api-platform/architecture/overview For a hands on experience and details overview, would recommend completing below course and labs: Course: API Design and Fundamentals of Google Cloud's Apigee API Platform              https://www.cloudskillsboost.google
Read more

API Design First approach: Implementing quick mock API's using swagger hub and postman

Image
Consider a scenario, where you as a API developer been ask to to implement the API's for a new mobile/web application team. The requirement is to implement a application to allow users to manage orders.         Mobile APP -> API infra->Backend enterprise system While implement a full working API might take days, you also do not want the application team to wait till the API's are ready. Thus, in most case we end up implementing the mock API's with sample scenarios. In this post, we will define an API using swagger openapi 3.0 yaml specification in swaggerhub. We will then use the definition to implement mock API's in postman. Even though postman provide option to define the API's, i liked swagger hub much better, it allow you to view your API on right side.  For this post I implemented API definition in swagger, please find the below link. https://app.swaggerhub.com/apis-docs/lopesrohan1988/ordermanagement/1.0.0 API Definition can be found at below link: https
Read more

Using Google Cloud Function to save JSON payload from Google Storage Event to Google Bigquery

Image
As part of this article we will learn below: Create the cloud storage bucket Use sample hierarchical JSON payload to auto create bigquery table Create google cloud function to listen to the object creation event on storage bucket and Fetch the JSON payload from the file and load in to Bigquery *We will be using Python for this activity. 1. Create the cloud storage bucket This is the simple task, you can either create the bucket using the console or gsutil command. For this exercise, we will use console. We will ignore some parameters like regional, bucket type, as we will not be using this bucket for large storage. Go to cloud storage -> Click on Create Bucket Give some unique name, you can use your Google Project ID+some name and click on create bucket. 2. Use sample hierarchical JSON payload to auto create bigquery table One of the quick way to create the Bigquery table to save the JSON payload with array is to first create the sample JSON payload and use it to create the bigquery
Read more