Building a Data Integration Team

افزودن به بوکمارک اشتراک گذاری 0 دیدگاه کاربران 5 (0)

Skills, Requirements, and Solutions for Designing Integrations

مشارکت: عنوان و توضیح کوتاه هر کتاب را ترجمه کنید این ترجمه بعد از تایید با نام شما در سایت نمایش داده خواهد شد.

ترجمه این کتاب مشاهده همه کتاب های ترجمه نشده

Skills, Requirements, and Solutions for Designing Integrations

Find the right people with the right skills. This bookclarifies best practices for creating high-functioning data integration teams, enabling you to understand the skills and requirements, documents, and solutions for planning, designing, and monitoring both one-time migration and daily integration systems.

The growth of data is exploding. With multiple sources of information constantly arriving across enterprise systems, combining these systems into a single, cohesive, and documentable unit has become more important than ever. But the approach toward integration is much different than in other software disciplines, requiring the ability to code, collaborate, and disentangle complex business rules into a scalable model.Data migrations and integrations can be complicated. In many cases, project teams save the actual migration for the last weekend of the project, and any issues can lead to missed deadlines or, at worst, corrupted data that needs to be reconciled post-deployment. This book details how to plan strategically to avoid these last-minute risks as well as how to build the right solutions for future integration projects.

What You Will Learn

Understand the “language” of integrations and how they relate in terms of priority and ownership
Create valuable documents that lead your team from discovery to deployment
Research the most important integration tools in the market today
Monitor your error logs and see how the output increases the cycle of continuous improvement
Market across the enterprise to provide valuable integration solutions

Who This Book Is For

The executive and integration team leaders who are building the corresponding practice. It is also for integration architects, developers, and business analysts who need additional familiarity with ETL tools, integration processes, and associated project deliverables.

02
02
Intermediate user level

04
02
Introduction (10 pages)“Without a systematic way to start and keep data clean, bad data will happen.” – Donato Diorio, CEO of RingLead

In this introduction, I talk about what data integration is and what it is not. I provide a working definition and how, in our text, I use migration and integration interchangeably. I describe why data integration matters, under what use cases it is most successful, and what can happen when integrations fail. I caution that readers need to be familiar with basic databases practices (how they work in collaboration with departments) and have a basic understanding of workflows and process handoffs. I also recommend that readers have some experience with business templates such as Requirements Documents and Sequence Diagrams.

Chapter 1: Integration Background (30 pages)

I start with a brief history of data migration, from its earliest days to the more modern times of cloud migration. Through this, readers will be able to grasp the advancement in architecture, speed, and complexity. We then discuss how integration is more of a process rather than a product and “owning a process” requires a different perspective that the more familiar “owning a product” that software development entails. We end the chapter with a discussion of integration approaches ranging from the one-time migration, nightly integration, listener services and hybrid approach.

1) Brief history

2) Process Ownership vs. Product Ownership

3) Integration Approaches

a. One-time Migration

b. Nightly Integration

c. Listener services

d. Scheduled services

e. Hybrid approach

Chapter 2: Key Terms (20 pages)

Integration developed from a combination of both technical and business mindsets and consequently has several terms that require some understanding. This section is designed to clarify what these terms mean and why they matter in the integration process.

Some of the terms to be discussed include

1) Metadata

2) Source

3) Target

4) Mapping

5) Extractions, Transformation, and Loading (ETL)

Chapter 3: Team Qualifications (20 pages)

The integration team consists of individuals who are technical engineers, business-focused analysts, great communicators, and experienced coders. While some teams are quite small (I’ve seen those with two people or less), others can be much larger (a typical size of 5-9 could be the norm). This chapter discusses the roles and responsibilities required to create a great integration team.

1) Data Integration Architect

2) Developer

3) Release Manager

4) Project Manager

5) Stakeholders

Chapter 4: Finding your Purpose: Project Deliverables (30 pages)

With a team structure firmly in mind, the next step is to determine what type of functional, business, and technical requirements should be captured and documented. Data integrations tend to be very fluid, and often, mappings can change multiple times before the business provides final signoff. If large datasets reveal inexplicable errors, developers must quickly research, code, and communicate workarounds to stakeholders. The best way to do this is through a combination of good business requirements, an understanding of who can support change requests, and the right documents to communicate the integration approach.

1) The purpose of Business Requirements

2) A good communication plan: Knowing your Points of Contact

3) Primary documents

a. High-Level Design Document

b. Source to Target Mapping

c. Sequence Diagram

d. Architecture Diagram

e. ETL Screenshots

Chapter 5: ETL (35 pages)

Depending on the type of integration required, developers and architects need to have experience with database services, cloud platforms, XML, and underlying communication protocols like REST and SOAP. Tying these technologies together requires the right ETL tool. This chapter discusses how ETL typically operates and includes a sample script to demonstrate the steps to build a simple service. We then go through the current ETL software leaders in the marketplace ranging from the least amount of experience (but with little flexibility) to the more advanced (but with plenty of bells and whistles). The key here is to help establish the right tool for the right job without having to learn a series of new technologies overnight.

1) An introduction to ETL

a. Extraction (Connectors, Part 1)

b. Transformation

c. Load (Connectors, Part 2)

2) Sample exercise

3) Popular Software

a. Jitterbit

b. Talend

c. Dell Boomi

d. Pentaho

e. SQL Server (SSIS)

f. MuleSoft

g. Informatica

h. Scripting languages (Python, Perl)

Chapter 6: Platform Automation (20 pages)

One of the hallmarks of a good integration design is that it should be repeatable. The initial load scripts should be tested rigourously on Develoment and Test environments, and daily processes should run continuously on Production Environments, changing where needed to fit the growing needs of the business. Much of this approach relates directly to a DevOps model and bears discussing as it relates to data integration.

1) Environment Builds

2) Running your pipeline through orchestrations

3) File Version Control

4) Testing

a. Back-End

b. Front-End

5) Deploying through release management

Chapter 7: Monitoring Results (25 pages)

The design is complete, the business is satisfied with the requirements, and the integration has gone live. The only thing left to do is to start monitoring the results of the integration. In this chapter, we implement a PDCA (Plan-Do-Check-Act cycle) to improve the integration output, providing daily success and error counts to users through emails and other notification channels. We discuss ways to identify Type I and Type II errors and making sure the owners of the data systems know how to resolve their issues once discovered. We end with a brief discussion on using the integration as a feeder into business intelligence, potentially using predictive analytics to discover gaps in the data that can lead to additional integration projects.

1) The Continous Improvement Model

2) Identifying Successes and Errors

3) Alerting Teams Through Different Notification Channels

4) Using Analytics to generate future integrations

Chapter 8: Building Outward: Marketing to the Enterprise (20 pages)

By this point, the integration team should have deployed at least a few projects and have earned accolades for their successful accomplishments. This celebration is no time to rest on their laurels, however. Before the excitement dies down, teams should reach out to departments across the enterprise, identifying future projects that make the best candidates for data integration. Part of this approach will involve advertising to make external teams aware of the work the integration team has done, performing educational activities such as Lunch-and-Learns, and become involved with building an Integration Center of Excellence (I will discuss this in some length). Understanding the goals of the enterprise for the upcoming financial year would also have value; with some research and creativity, the team can construct an integration data roadmap. This visual summary, similar to a product roadmap, maps out the vision and direction of integration offerings, which departments they serve, and the estimated time it would take to complete.

1) Advertising

2) Education

3) Building an Integration Center of Excellence

4) Data Roadmaps

Appendix A: Sample Templates

There will be some follow-up in the Appendix including the code and sample templates (which are also stored in GitHub). I’d also like to include some additional weblinks to the ETL software (we might want to move that section over to the Appendix if it doesn’t fit with the rest of the book).

13
02
JarrettGoldfedder is the founder of InfoThoughts Data, LLC, a company that specializes in data management, migration, and automation. He has significant experience in both cloud-based and on-premise technologies and holds various certificates in Salesforce Administration, Dell Boomi Architecture, and Informatica Cloud Data. He also served as a technical reviewer of the Apress book by David Masri titledDeveloping Data Migrations and Integrations with Salesforce: Patterns and Best Practices.

18
02
Find the right people with the right skills.This bookclarifies best practices for creating high-functioning data integration teams, enabling you to understand the skills and requirements, documents, and solutions for planning, designing, and monitoring both one-time migration and daily integration systems.

The growth of data is exploding. With multiple sources of information constantly arriving across enterprise systems, combining these systems into a single, cohesive, and documentable unit has become more important than ever. But the approach toward integration is much different than in other software disciplines, requiring the ability to code, collaborate, and disentangle complex business rules into a scalable model.Data migrations and integrations can be complicated. In many cases, project teams save the actual migration for the last weekend of the project, and any issues can lead to missed deadlines or, at worst, corrupted data that needs to be reconciled post-deployment. This book details how to plan strategically to avoid these last-minute risks as well as how to build the right solutions for future integration projects.

You will:

Understand the “language” of integrations and how they relate in terms of priority and ownership
Create valuable documents that lead your team from discovery to deployment
Research the most important integration tools in the market today
Monitor your error logs and see how the output increases the cycle of continuous improvement
Market across the enterprise to provide valuable integration solutions

19
02
Other technical guides focus on the use of a specific integration tool (ETL, scripting) rather than the actual framework itself—this book offers a strong foundation for promoting these tools and enhancing teamwork

Educate business users on the nuances behind assigning the right staff to the right tasks

Design the integration team with a strategic approach from the outset of a project rather than a last-minute afterthought

06
05
300
01
https://covers.springernature.com/boo...

01
01
https://www.springer.com/9781484256527

01
http://www.springer.com/

01
Springer Nature Imprint
APR
Apress

01
01
APR
Apress

01
05
5235327
Apress

Berkeley, CA
US
04
20200228
2020

01
WORLD

08
515
gr

01
254
mm

02
178
mm

27

03
9781484256534

15
9781484256534

01
ISBN-13 hyphenated
978-1-4842-5653-4

DG

Apress
01
ROW
IP
23
20200228

02
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
54.99
AUD
AU
20191206

01
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
49.99
AUD
AU
20191206

02
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
39.00
CHF
CH
R
2.5
20191206

02
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
35.30
EUR
DE
R
7
20191206

02
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
34.80
EUR
FR
20191206

02
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
34.31
EUR
IT
20191206

02
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
35.96
EUR
NL
20191206

02
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
36.29
EUR
AT
R
10
20191206

01
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
32.99
EUR
ROW
20191206

01
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
27.99
GBP
GB
20191206

01
Recommended Retail Price

01
BIC discount group code
ASPVBT

02
Product discount group
SPVBT

02
2350.00
INR
IN
20191206

Apress
01
US
02
Y
IP
23
20200228

01
Recommended Retail Price

01
BIC discount group code
ADGNY14

02
Product discount group
DGNY14

02
32.99
EUR
ROW
20191206

01
Recommended Retail Price

01
BIC discount group code
ADGNY14

02
Product discount group
DGNY14

02
37.99
USD
US
20191206

لغو

گزارش تخلف

مغایرت عنوان کتاب با توضیحات و تصویر جلد
عدم رعایت حقوق نشر طبق قوانین جمهوری اسلامی ایران
محتوای مستهجن
توهین به قومیتها و محتوای نژادپرستی
توهین به اقیلیتهای مذهبی
اهانت به مقامات سیاسی و مذهبی
اشاعه ی کودک آزاری
سایر موارد

بستن

فرمت کتاب

ebook

زبان

english

تاریخ انتشار

2020

نویسنده

Jarrett Goldfedder

ناشر

Apress

شابک

9781484256527

20,000 تومان

خرید کتاب مطالعه نمونه درخواست ترجمه کتاب

خرید کتاب Building a Data Integration Team

Building a Data Integration Team

گزارش تخلف

فرمت کتاب

ebook

زبان

english

تاریخ انتشار

2020

نویسنده

ناشر

شابک

کتاب های مرتبط

Microsoft SQL Server 2012 Integration Services

Computational Methods for Data Evaluation and Assimilation

Pentaho Data Integration Cookbook Second Edition

Big data integration theory : theory and methods of database mappings, programming languages, and semantics

Geostatistics for Seismic Data Integration in Earth Models

Information reuse and integration in academia and industry

Creating Value from Data Sharing

Data Integration Life Cycle Management with Ssis

Network Medicine

Pentaho Data Integration Quick Start Guide

Data Integration in the Life Sciences

Managing Data in Motion

Model-Based Software and Data Integration

Master Data Management in Practice

Guidelines for Application Integration

Building a Data Integration Team

Polybase Revealed

Quantitative Intertextuality

Data Integration in the Life Sciences

Pentaho Kettle Solutions

Big Data Integration

دیدگاه کاربران