Extract-Phase |
Scroll Previous Topic Top Next Topic More |
In the extract phase the app MPM_Process_01_QVDBuilder is employed. The app extracts tables from a data base like SAP and stores them into a dedicated folder. It may occur that tables are already extracted but not yet in .qvd-format or stored in multiple files for sizing reasons. Files that are not ready for further modelling, because they are disperse or not in .qvd-format are stored in 01_Extract/extract and then converted by the app MPM_Process_01_QVDBuilder into .qvd-files that are stored under 01_Extract/transformed.
QVD is a native Qlik format written and read only by Qlik Sense® or QlikView. The file format is optimized for speed when reading data from a script but it is still very compact. For more information please refer to the Qlik Sense® help.
To create the event “Create Purchase Order” from an SAP data base, we need to gather all related tables. As you know, an event has three key information that are needed to do process mining: CaseID, event timestamp and event definition. The CaseID is the same for all events, but the timestamp is unique for each event.
1.First, we should think on how to define an event and which tables will deliver the required information like timestamp or Case ID. Eventually, there is need to define a special Case ID. In this example the definition of a case ID in a purchase process can be the following:
•CaseID: InvoiceNr_Client_CompanyCode_FiscalYear
2.Second, we need to extract tables from which we can combine the fields required to create the Case ID and to get the timestamps. For the event “Create Purchase Order” we use these tables:
•Change log: CDHDR, which is used to store Change document header data.
•Change log: CDPOS, which is used to store Change document items data.
•RSEG: It is used to store Document Item: Incoming Invoice data
3.Third, as different tables can contain the same field names (i.e. MANDANT), we qualify the tables with the function MW_qualifyFieldNames('TableName','Qualifier','Unqualify') to avoid synthetic keys and circular references.
4.Fourth, the extracted table is stored as .QVD file.
5.And finally we dropped the table to clear the RAM for better performance.
//Name the table and then insert the load script with the data connection dialog on the right site
CDHDR:
LOAD |
MANDANT & '_' & OBJECTCLAS & '_' & OBJECTID & '_' & CHANGENR as %CDHDR_PKEY, |
|
timestamp(date(UDATE) + time(UTIME)) as AETST, |
|
MANDANT, |
|
OBJECTCLAS, |
|
OBJECTID, |
|
CHANGENR, |
|
USERNAME, |
|
UDATE, |
|
UTIME, |
|
TCODE, |
|
PLANCHNGNR, |
|
ACT_CHNGNO, |
|
WAS_PLANND, |
|
CHANGE_IND, |
|
LANGU, |
|
VERSION, |
|
_DATAAGING; |
SELECT |
MANDANT, |
|
OBJECTCLAS, |
|
OBJECTID, |
|
CHANGENR, |
|
USERNAME, |
|
UDATE, |
|
UTIME, |
|
TCODE, |
|
PLANCHNGNR, |
|
ACT_CHNGNO, |
|
WAS_PLANND, |
|
CHANGE_IND, |
|
LANGU, |
|
VERSION, |
|
_DATAAGING |
FROM CDHDR
WHERE OBJECTCLAS EQ 'EINKBELEG' AND UDATE GE '$(mvDateFilter)';
// The function MW_qualifyFieldNames is used to qualify the table MW_qualifyFieldNames('TableName','Qualifier', 'Unqualify')
call MW_qualifyFieldNames('CDHDR','CDHDR', '%*');
// The qvd is stored in the respective folder structure (defined in section main)
Store CDHDR into [$(DataConnection)/ProcessAnalyzer_$(FDescription)/01_Extract/transformed/CDHDR_EINKBELEG.qvd] (qvd);
// Table is dropped to clear RAM (performance)
Drop Table CDHDR, temp;