Please enable JavaScript.
Coggle requires JavaScript to display documents.
BD in Ese - Coggle Diagram
BD in Ese
3 BD Entreprise model
data sources : various sources lile App virtualized / bare Metal , cloud , click streams , social media , events , sensor data , mobility trends , logs
data storage and processing : BD(Nosql , real time capture , read + update)
traditional db
storage(san/ NAS)
data analysis : Uses Big Data framework (Hadoop) for storing and analyzing data.
Traditional enterprise data models have become costlier and more complex due to big data needs.
Rapid changes have altered how big data is stored, analyzed, and accessed.
New models use scalable, shared-nothing architecture, presenting new challenges.
Traditional models now include new frameworks designed for big data needs.
These systems must integrate with current business models, data strategies, and network infrastructures.
2 Data processing and analytics: old way
Traditional Approach
Data processing for analytics follows a static and predefined blueprint.
Data Sources
Data is sourced from enterprise applications like CRM, ERP, and financial systems.
Data Integration
ETL (Extract, Transform, Load) tools are used to extract, transform, and load data into a staging area.
Goal:
Ensure data quality and normalization before modeling it into structured rows and tables.
Data Warehousing
Cleansed and modeled data is loaded into an enterprise data warehouse.
4 Building a BD platform
obj principale => intégrer facilement les BD avec les data d'Ese afin de réaliser des analyses approfondies
Infrastructure needs: Data acquisition, organization, and analysis(analyse des data)
a) Big data acquisition
Big Data acquisition requires systems with low latency for fast data capture and queries.
Systems must handle high transaction volumes in a distributed environment.
Flexible structures are essential for varied data formats (text, images, etc.).
NoSQL databases are ideal because:
They scale easily with growing data.
They support unstructured, dynamic data without requiring a fixed schema.
b) Organize BD
In traditional data warehouses, data is moved and organized in a central location, but Big Data is organized at its storage location to save time and cost.
Infrastructure must:
Process data at its original location.
Handle high data throughput, often in batches.
Support varied data formats (structured and unstructured).
Hadoop organizes and processes large data directly on its storage cluster.
HDFS(sys de fichier) stores data long-term (e.g., web logs).
then MapReduce processes data to extract insights (e.g., browsing sessions).
Results are sent to relational databases (RDBMS) for further analysis.
c ) Analyse BD
Integration with Traditional Data: Combines new data with older data to gain fresh insights on old problems.
Example: Analyzing inventory data and local event calendars for an automated vending machine can optimize product selection and replenishment scheduling.
Advanced Infrastructure: Supports complex analysis like statistical analysis, data mining, and AI for faster decision-making.
Big Data is not just merging social and enterprise data :
Integrating hard-to-capture or ignored data.
Processing unstructured data formats (e.g., log files).
4 BD components
deux principaux bloc de constructions
Hadoop
Provides storage with a distributed, shared-nothing file system.
Enables data analysis via MapReduce.
Nosql
Captures, reads, and updates large volumes of unstructured data in real-time.
Manages data from click streams, social media, log files, event data, mobility trends, sensors, and machines.
Social Data : unstructured content (text, audio, video,images)from the web , social media and public channels
Ese Data : structured data within the compony (customer info , products , transactions ) used for operations
=
+
BD: combining social + entreprise data provides deeper insights, leading to bettter products => customer satisfaction / higher revenues / profitability
Businesses leverage(tastafid) this to gain deeper insights and enhance competitiveness.
1
explication
