Please enable JavaScript.
Coggle requires JavaScript to display documents.
Data Management & Govenance, Sources - Coggle Diagram
Data Management
& Govenance
Data Quality &
Management
Master & Reference data management - #2
Identifying & resolving issues
Indentify and resolve duplicates
Catalogue where we have
multiple sources with conflicts
Identify business rule failures
Exception reporting of
quality issues - C
Processes for resolving inconsistent
records within a dataset
Processes for resolving inconsistent
records between datasets
Record and report
data quality issue trends - C
Establishment of data quality standards
for key master and reference data (completeness, uniqueness, timeliness, validity, accuracy, and consistency) - C
Processes for investigating and resolving
ongoing data quality issues - C
Mechanism to hold business rules in a way that they can be utilized to measure and manage data quality
Security and control of master data - E
Capture systems of record
and sources of truth for each
master data record - B & E
Guidance for maintaining data
Business rules repository (BC added: for where rules are applied manually) - B
Define business rules for the purposes
of maintaining data and identifying when
data quality issues exist - B (Business Rules
Repository)
Define taxonomies - B
Agreement on Taxonomies :fountain_pen:
Define nomenclature
Training materials - B :pencil2:
Event / transaction data - #2
Process documentation
for capturing data
Most of the "Identifying & resolving issues"
points apply to event / transaction data too.
Semi-structured data (includes spreadsheets,
SHAREPOINT LISTS(?) - #1 ??
Processes for loading
& maintaining data - including
version control & logging
Strong controls where used in
business critical processes
Policies and guidelines for
managing this data
Single source of data manipulation to facilitate data interoperability - F
Provide a mechanism for recording data not maintained in a source system i.e. data maintained directly in the DWH currently - #2
Data Privacy & Security
not mentioned,
should be part of #3?
Processes and controls for
managing access
Partitioning / segmenting data
where needed to provide security controls
Object level, column level
and record level security
Maintaining privacy policies and guidelines
Ensuring users have access
to data to do their job (appropriate
security controls)
Access control considerations for non-SFF users
Data Architecture
Storage
Archiving infrequently used data - #2
Ability to access archived data
Storing data in an appropriately performant & costing repository based on how the data will be used - #1
Able to see historical versions of records e.g. Nav PO example - E
Audit trails & logs (structured and
non-structured) - B
Data "freshness" matches use cases - #1
Storing structured data - #1
Store event / transaction data
Master / reference data
Archiving / data retention rules - #2
Rules are defined
Rules are applied
Storing semi-structured data - #1
Include the ability to ingest and store Excel files that are frequently updated and are used in business critical processes
Structuring Data - #2
Provide a high-level view of data entities (CDM) - B
Defined entity attributes etc (LDM - B)
Define how entities will be used physically
Processes, guidelines and stds for
managing CDM and LDM - B & C
Create and maintain a Conceptual
Data Model - B
Create and maintain a Logical
Data Model - B
Apply the Microsoft Common Data Model
Define metadata to be captured for
entity, relationship and attribute levels
Data models meet our market
and customer requirements - C
Data Integration - #4
Patterns - B
Review - B
Metadata
Provide a data catalogue - #2
User able to search for data assets to meet their needs
Data lineage
Ability to find all data assets from a source
Tracing data back to source / owner from a data asset - E
Able to search for data assets, sources, processes
based on columns & other parameters
Data tagging
When to be disposed -
e.g. X years after last use :champagne:
Critical or sensitive in nature
Provide metadata within data usage e.g.
glossary & lineage info within a report
Provide a Data Glossary for use within a wider Business Glossary - B (extended - based on use of word 'data' vs 'business') - #2
Provide a data dictionary - #2
Definitions and other attributes for data entities - E
Metadata repository - B - #2
Include Business Metadata
Business rules
Data Stds
Data Models
Include Operational Metadata
Log files
Archiving / data retention rules
Report access patterns
Include Technical
Metadata
Physical database definitions
Data lineage
Access permissions
Tagging and categorising Metadata objects
Users must be able find metadata artefacts e.g. links on OurSource that users that will show in a key word search
Must be able to be readily maintained to a good quality by Super Users / SMEs - for this asset to be sustainable
Data Lifecycle
Managing metadata - C
Ensuring that data / information is
aligned to business model / strategy - C
Processes to review requirements
against business model / strategy
Processes to assist users with understanding
data assets & how to reduce time collating/ validating data - C
Processes to work with improvement projects to help with value realisation and cross-functional risk management - C
Processes to identify incomplete and dirty data (not just data quality) - C
Data and Info Retention and Disposal Policy and Scehdule and suporting documentation (to cover data and info in any format or location) :fountain_pen:
Out of scope?
Managing Business Information / Content - B
Content repository - B
Policies & procedures - B
Strategy - B
Managing systems
and their content
Promapp business process maps including
QMS documentation
Butchery specification
Learning Management Systems
Storing unstructured data - Not explicitly
included in evaluate papers
Include the ability to ingest and store videos, photos, documents and other files that we need to mine for data
Include the ability to ingest and store ad-hoc excel, CSV and other data files
Include the ability to ingest and store streamed data
Securing data within source systems?
Data Governance - #3
Governance Model - #3
Governance Frameworks
Stewardship framework, roles and responsibilities
policies, frameworks
standards
procedures - methodssteps, techniques to produce certain outcomes, supporting artifacts, etc
Monitoring and Assessment
Governance Issue Management - #3
change management
compliance - issues management
managing conflicts with policies, procedures, rules, definitions, ownership, etc
issues relating to conforming to policies standards, procedures, etc
data security and identity issues, including breach investigations
data issues management
data quality
contracts - relating to data suply, sharing, etc
Authority - regarding decision rights, procedures, etc
Sources
E.
Stellar
F.
Datacom
B. Data Description Project High
Level Requirements
C Evaluate 2 - Data Management Foundations
A. Strategy Document