This link is mainly based on business-based data cleaning and logical processing of data usage scenarios, including offline data development and real-time data development, which is equivalent to a data processing factory, processing based job function email list on synchronized data sources to form a highly available data model. The development kit is relatively large and can be independently formed into a separate product module. At the same job function email list time, model construction specifications can be integrated into the verification process of task development. Do more pre-checks instead of just relying on after-the-fact governance.
For example, providing process modeling or job function email list data processing tools such as dataphin 3. Data asset management Asset-based management: The data processed by the data factory needs to be sorted into categories and labelled with various specifications before it can be used by downstream consumers. Asset management mainly uses data maps job function email list to query and retrieve data tables, maintain and query metadata information, and provide users with convenient data guidance capabilities.
Data blood relationship: It is a data link relationship that runs through the whole process of data from entering the lake to the business terminal. job function email list First, it can facilitate the investigation of the ins and outs of the data production process, and provide guidance for code review logic. In addition, based on blood relationship, downstream notification can job function email list be achieved when data is abnormal, and when downstream applications are not used, data can be managed with one click, and storage and computing resources can be released