Financial Data Integration & Containerization Platform

This is a cloud-native, AI-powered integration system designed to automate financial data mapping, transformation, and reconciliation between the Planto Wealth Management Application and proprietary financial calculator deriving from Fuego. The solution is built using Google Cloud services and containerized with Docker for maximum portability and scalability.

It started with a motivation to improve how I track and plan my own finances as a retail investor in Hong Kong, I teamed up with my brother, who studies Information Systems at HKUST, to develop a simple wealth management workflow for Hkers.

Project Overview

The Challenge

Managing personal finances across multiple platforms often involves:

• Manual data entry and reconciliation
• Inconsistent data formats and schemas
• Time-consuming categorization and labeling
• Limited integration capabilities between financial tools

Our Solution

Engineered a comprehensive solution that:

• Automates data extraction, transformation, and loading (ETL) processes
• Leverages AI for intelligent data categorization
• Provides real-time synchronization between platforms
• Ensures data accuracy through automated validation
• Scales efficiently with containerized microservices

Technical Implementation

System Architecture

• Frontend: Custom dashboard for monitoring and control
• Backend: Microservices architecture with Google Cloud Run
• Data Processing: Custom ETL pipelines with schema mapping
• AI Integration: NLP for intelligent categorization
• Containerization: Docker for consistent environments
• CI/CD: Automated deployment with Cloud Build

Key Components

1. Data Ingestion Layer

   • Secure API connections to Planto and other data sources
   • Automated data extraction on scheduled intervals
   • Support for multiple data formats (CSV, JSON, API responses)

2. Transformation Engine

   • Schema mapping and normalization
   • AI-powered categorization using NLP
   • Custom business rules for financial data

3. Storage & Processing

   • Google Cloud Storage for raw data
   • Cloud SQL for processed data
   • In-memory processing for real-time operations

4. Containerization

   • Docker containers for each microservice
   • Docker Compose for local development
   • Container Registry for version control

5. Deployment

   • Google Cloud Run for serverless deployment
   • IAM for secure access control
   • Monitoring and logging with Cloud Operations

My Role & Contributions

As the lead developer and architect, I was responsible for:

• System Design: Architected the entire cloud-native solution
• Development: Implemented core functionality in Python and JavaScript
• AI Integration: Added NLP capabilities for intelligent data processing
• DevOps: Set up CI/CD pipelines and container orchestration
• Documentation: Created comprehensive documentation for maintenance

Technical Stack

• Cloud Platform: Google Cloud Platform
• Containerization: Docker, Docker Compose
• Orchestration: Google Cloud Run
• Storage: Cloud Storage, Cloud SQL
• CI/CD: Google Cloud Build
• Languages: Python, JavaScript (Google Apps Script)
• AI/ML: NLP for data categorization
• Monitoring: Cloud Operations, Cloud Logging

Impact & Results

• Efficiency: Reduced manual data entry time by 80%
• Accuracy: Improved data consistency to 99.9%
• Scalability: Designed to handle growing data volumes
• Cost-Effective: Serverless architecture minimizes operational costs
• User Experience: Simplified financial management for end-users

Future Enhancements

• Add support for additional financial platforms
• Implement advanced analytics and forecasting
• Enhance AI capabilities for smarter categorization
• Develop mobile application for on-the-go access
• Add multi-currency support

If you are in HK and would like to try out our workflow, feel free to request it from us @sean021453@gmail.com for a free demo.