Please enable JavaScript.
Coggle requires JavaScript to display documents.
Data Management - Coggle Diagram
Data Management
02.Analytics Enablement
Data Curation Framework
Data storage
File
Blob
Tables
Data processing
Configuration
Orchestration
Logging
Alert & Notification
Email
Slack
Azure DevOps
Data archiving
Timestamping
Housekeeping
Data Modelling Review
Algorithm selection
Test & Validation
Deployment
Data Curation Guideline
Requirement
Use case
Business user
Analytical user
Data source mapping
Internal
External
Third party
Vendor
Collection methods
Offline
API
File
DB Connection
Scraping
Sensors
Mobile
Infrastructure design
Development
Testing
Deployment
Coding
Code Review
Git management
Code version conflict solutions
Operating Model, Way of Working & SOP
Modular
Detection
Diagnostics
Homepages
Simulator
Alert
Context
01.Data Enablement
02.Data Ingestion
Data Ingestion Framework
Scalability
Load Balancing
Lose Coupling
Data Provenance / Data Lineage
Incremental Load
Generalization
Config Based / Parameterized (Not Hard Coded)
Dictionary Based
Reliability
Monitoring
CI/CD
Security
01.Use Case Management (BRD & DDPA)
Cost & Benefit Analysis
Template
Data Share Agreement
Application Owner, Data Owner & Data Steward Approval List
DSA Docuement
Data Sourcing
Data Custodian Collaboration
Data Dictionary & Mapping Document
Business Requirement Document or Use Case Template
04.Data Architecture
Data Landscape Management (Tracking & Monitoring)
Development Tracking
Production Tracking
Initial Ingestion
CDC Dashboard
Streaming, NRT, Realtime Monitoring Dashboard
Data Share Agreement
DSA for Data Owner
DSA for Consumption
Data Catalog Management
Data Dictionary
Data Categorization, Classification and Tagging
Business Glossary, Business Rules & Data Hierarchy
Data Lineage
Information Life Cycle Management
Business Owner
Data Owner & Steward
Application Owner
Document, Content, Information, Insight Management
Master Data Management
Customer & Consumer
Material or Product
Hospital
Professional Healthcare
Geographical & Area
Knowledge Base for AI
BioMetric
Gondola Good Face Reference
Others
CI/CD & Code Review
CI/CD & Code Review Guideline
Tools
SOP
Enterprise Data Model
Data Model & Design
Conceptual Data Model (CDM) - Business Driven
High level, non technical arrangement of Data Domains / Sub-domains, their business concept and the relationship between them
This modeling is dependent of any underlying technological platform
Logical Data Model (LDM) - IT/Business Driven
Include Data Element as attributes/characteristics within the defined Business Concept and their relationship in a hybrid technical and business terminology
it is a intermediary model that remains independent from physical implementation
Physical Data Model (PDM) - IT Driven
Database specific models that represents the physical data elements and their relationship
Data Leveling Design
L1 - Data Domain
Data Domain is the high-level representation of a group of data used by a business division within the organization.
Each Data Domain might contain sub-domains, which contains Business Concepts related to a specific purpose or function (customer data, employee data, product data, etc.).
Domains are important as they are used to group and classify sets of data from a high level business perspective.
L2 - Sub-Domain
Data Domains are hierarchically organized optionally by logical divisions called Sub-Domains.
Data Domains and Sub-domains organize the Business Concepts within the data model.
L3 - Business Concept
A Business Concept is an essential contributor to build the structure of the data.
A Business Concept could represent a person, place, thing or concept of interest to an organization in the real word. Each Business Concept type has a unique, singular name, for example, customers, products, sales, etc.
L4 - Data Element
Data Elements are characteristics! attributes of the Business Concept, used to define or conceptually represent one or more physical data elements.
L5 - Physical Data Element
Physical Data Elements are characteristics of the physical data that is represented in a system / application, they are assigned to at least one Data Element within a data model.
Data Access Management
DAM for Published Layer
DAM for Curated Layer
03.Data Refreshment (CDC)
CDC Strategy
Full Load
Delta Load
Streaming, NRT or Real Time
CDC Monitoring
CDC Loose Coupling Review (Semester)
05.Data Stitching / Curation
Raw Data Layer Management (~ Data Ingestion) - Collab with ISIT
Cleaned Data
Data Quality Management
Completeness
Data Source vs Landing Zone (row sum)
Data Sampling random check
Mandatory Data Element & Null Value check
Conformity (right format?)
Data Type: source vs landing zone
Data Format: source vs landing zone -> dd/mm/yyyy
Data Length
Timeliness
Daily
Hourly
Monthly
NRT, RT, Streaming
Standardized Data
Data Quality Management
Master Data
Uniqueness
Consistency
Business Rules Agreement Matching Process
Cross Department
Cross System
Accuracy: reflect real world
Transactional Data (Fact Data)
Completeness
Conformity
Accuracy
Business Rules Management
Staging Layer
DQM
House Keeping
Data Retention Management
Stitched Detail Data
Data Mart Management
Use Case Related
How to Share
Data Access Management
Data Lab Collaboration
Advanced Analytics
AI & Machine Learning
CI/CD & Code Review
Deployment Management
03.Technology Hub
Data Sourcing
FTP/SFTP
API
Gateway
Data Integration
Azure Data Factory
Informatica Data Integration
Data Base & Storage
Azure Data Lake Gen2
Container or Blob Storage
Structured
Unstructured
Tables
Structured
Azure Cosmos Db
Azure SQL Server DB
Data Governance
Master Data Management (MDM) Solution
Informatica MDM
Data Quality Management Solution
Informatica BDQ
Utilize Power BI
Data Profiling & Monitor
Enterprise Data Catalogue Solution
Informatica
Manual Excel
Template
Knowledge Management
Microsoft SharePoint
Confluence
Data Share Agreement (DSA) Management
Confluence
Service Now
Data Computing
Data Bricks
Python / Anaconda Python
Hive
Spark
04.Framework & Way of Working
Communication Strategy
Release, News Update
Technical Debt update
Frameworks
Data Sourcing
Data Ingestion Framework
Data Orchestration Framework
Framework alignment with CDO
Wow
Internal
SOP
SOP Data Ingestion
SOP Data Curation
External
SOP & Segregation with ISIT
Alignment with Global CDO
05.Industrialization
Users
Internal
2nd Layer (Distributor or Retailer)
3rd Party Layer
Publish and Presentation Layer Methodology
API
Visualization or BI Tools
Database to Database, Direct Connection or Gateway
Power App
DGov - DSA Management