Course : Microsoft Fabric data engineer
Practical course - 3d
- 21h00 - Ref. FDN
|
Download in PDF format
Share this course by email
 | Set up a work environment adapted to the needs of an analytical project |
| Understanding the impact of parameters (Spark, Domain, OneLake...) on performance |
| Adapt loading models to project needs |
| Understand different batch ingestion methods and know how to transform and clean data |
| Master orchestration tools and automate according to events |
| Understand real-time data processing concepts |
| Ensure operational follow-up of treatments |
| Quickly diagnose errors |
| Integrating DevOps practices in a data environment |
| Ensuring data security |
Intended audience
Data engineers, data architects, developers or data consultants.
Prerequisites
Knowledge of relational databases and SQL, knowledge of data programming and general knowledge of data engineering.
Practical details
Exercise
Discussions, practical work.
Teaching methods
Active.
Course schedule
1 Introducing the Fabric platform
- Understanding the Microsoft Fabric ecosystem and its key components
- Identify typical use cases in data engineering
- Situating Fabric in a modern data architecture
2 Configuring Microsoft Fabric workspace parameters
- Create and configure a workspace for the needs of a data project
Hands-on work
Create and configure workspace parameters. Justify configuration choices according to context.
3 Design and implement loading models
- Understanding the challenges of full and incremental loading
- Preparing data for a dimensional model
- Defining a loading model for streaming
- Lakehouse, warehouse and eventhouse
Hands-on work
Design the right model for the job. Efficiently manage incoming flows.
4 Ingest and transform batch data
- Choice of storage type
- Use of dataflows, notebooks, KQL and T-SQL
- Shortcut management, mirror duplication
- Ingestion with pipelines
- Transformation with PySpark, SQL
- Denormalization, aggregation
- Management of duplicates, missing or out-of-date data
Hands-on work
Define an operational batch ingestion strategy. Apply the right transformations.
5 Orchestrating processes
- Pipeline or notebook
- Creating event schedules and triggers
- Integration of parameters and dynamic expressions in notebooks/pipelines
Hands-on work
Identify the right orchestration method. Implement a pipeline with event-based automation.
6 Real-time ingestion presentation
- Use native storage or shortcuts
- Processing with Eventstreams, Spark structured streaming, KQL
Hands-on work
Master the concepts and tools associated with real-time ingestion.
7 Monitor Fabric resources
- Monitoring ingestion, transformation, model updating
- Alert configuration
Hands-on work
Identify the right KPIs. Respond to alerts.
8 Identifying and resolving errors
- Identifying errors in pipelines, streams, notebooks, eventhouses, eventstreams, T-SQL
Hands-on work
Detect, analyze and correct errors.
9 Implementing lifecycle management in Fabric
- Git integration overview
- Implementation of database projects
- Creating and configuring deployment pipelines
Hands-on work
Create an operational deployment pipeline. Identify the impact of a new version.
10 Configuring security and governance
- Implementing workspace-level access controls
- Configure access controls by resource, row, column, object
- Setting up dynamic masking
- Activating and using workspace logging
Hands-on work
Apply fine-grained access strategies. Demonstrate dynamic masking. Analyze logs pertinently.
PARTICIPANTS
Data engineers, data architects, developers or data consultants.
PREREQUISITES
Knowledge of relational databases and SQL, knowledge of data programming and general knowledge of data engineering.
TRAINER QUALIFICATIONS
The experts leading the training are specialists in the covered subjects. They have been approved by our instructional teams for both their professional knowledge and their teaching ability, for each course they teach. They have at least five to ten years of experience in their field and hold (or have held) decision-making positions in companies.
ASSESSMENT TERMS
The trainer evaluates each participant’s academic progress throughout the training using multiple choice, scenarios, hands-on work and more.
Participants also complete a placement test before and after the course to measure the skills they’ve developed.
TERMS AND DEADLINES
Registration must be completed 24 hours before the start of the training.
ACCESSIBILITY FOR PEOPLE WITH DISABILITIES
Do you need special accessibility accommodations? Contact Mrs. Fosse, Disability Manager, at psh-accueil@orsys.fr to review your request and its feasibility.
Data engineers, data architects, developers or data consultants.
PREREQUISITES
Knowledge of relational databases and SQL, knowledge of data programming and general knowledge of data engineering.
TRAINER QUALIFICATIONS
The experts leading the training are specialists in the covered subjects. They have been approved by our instructional teams for both their professional knowledge and their teaching ability, for each course they teach. They have at least five to ten years of experience in their field and hold (or have held) decision-making positions in companies.
ASSESSMENT TERMS
The trainer evaluates each participant’s academic progress throughout the training using multiple choice, scenarios, hands-on work and more.
Participants also complete a placement test before and after the course to measure the skills they’ve developed.
TEACHING AIDS AND TECHNICAL RESOURCES
• The main teaching aids and instructional methods used in the training are audiovisual aids, documentation and course material, hands-on application exercises and corrected exercises for practical training courses, case studies and coverage of real cases for training seminars.
• At the end of each course or seminar, ORSYS provides participants with a course evaluation questionnaire that is analysed by our instructional teams.
• A check-in sheet for each half-day of attendance is provided at the end of the training, along with a course completion certificate if the trainee attended the entire session.
• The main teaching aids and instructional methods used in the training are audiovisual aids, documentation and course material, hands-on application exercises and corrected exercises for practical training courses, case studies and coverage of real cases for training seminars.
• At the end of each course or seminar, ORSYS provides participants with a course evaluation questionnaire that is analysed by our instructional teams.
• A check-in sheet for each half-day of attendance is provided at the end of the training, along with a course completion certificate if the trainee attended the entire session.
TERMS AND DEADLINES
Registration must be completed 24 hours before the start of the training.
ACCESSIBILITY FOR PEOPLE WITH DISABILITIES
Do you need special accessibility accommodations? Contact Mrs. Fosse, Disability Manager, at psh-accueil@orsys.fr to review your request and its feasibility.
Dates and locations
Dernières places
Date garantie en présentiel ou à distance
Session garantie