Generic Extraction
SDN Articles - BI Extraction
List Of Articles:
1) DB Connect
2) Extraction from R3
2.a) COPA Extraction
2.b) Generic Extraction
2.c) LO Extraction
3) Flat File Loading
4) UD Connect
5) XML Services
The Setup Tables are the objects from which the Business Warehouse system is going to extract data for Full loads and Initialization on LO Cockpit DataSources. Not just "LO DataSources" because there are some that don't use this feature.
When a Full extraction or a Delta Initialization is going to be performed, the Setup Tables need to be filled with all the historical data to be loaded to the Business Warehouse system.
Their content can be deleted any time, without affecting the R/3 transaction data. In fact, after the full/Init load has been performed, it is highly likely that the information contained in the Setup Tables is already not up-to-date anymore. Therefore, it would be no harm at all to delete their content. You may think of it as a "PSA" on the R/3 side.
If the load performed was an Initialization, the next records to be extracted will be sent to the Delta Queues.
A Myth
It isn't 100% true to say that the reason for the Setup Tables is to ensure the extractors won't need to access the always busy application tables (like EKKO or VBAK). The application tables are still read during the setup running.
Setup Tables Nomenclature
The base nomenclature for these tables is MC*SETUP. It may not be easy to find the actual name you may be looking for, but if you display some in se11/se16 you may figure it out by the description.
I created the following Wiki page with the Setup Tables names:
BW Setup Tables List
If you need to quickly find the name of a name, just Control+F this Wiki!
Procedures involving the Setup
Well, it does sound pretty simple if you consider only the purpose of the Setup Tables. The part which would turn this subject eligible for a blog post is how to properly handle these tables in various possible activities.
I have composed a list of step-by-step explanations for some procedures that, in a way or another, end up involving Setup Tables somehow. I'll also mention what are the common consequences of not performing these activities correctly.
Deleting Setup Tables Content
1- To delete the Setup Tables, you may go directly to LBWG or choose the following IMG option (transaction SBIW).
IMG path:
- Data Transfer to the SAP Business Information Warehouse
- Settings for Application-Specific DataSources (PI)
- Logistics
- Managing Extract Structures
- Initialization
>>Delete the Contents of the Setup Tables
2- Select the application for which you want the Setup Tables to be deleted.
3- Click on "Execute" or press F8.
How can you check if the content was properly deleted?
You may check the correspondent Setup Table or perform an extraction check for the relevant extractors in RSA3.
Filling Setup Tables
1- Before filling the Setup Tables, you need to ensure at least 1 extractor for the application is active in LBWE.
2- To enter the transactions to perform the setup, you may either go directly to one of the following transactions, or choose one of te options from the SBIW menu displayed below.
Application | Transaction | Component | |
02
|
Purchasing
|
OLI3BW
|
BW-BCT-MM-PUR
|
03
|
Inventory Controlling
|
MCNB
OLI1BW
OLIZBW
|
BW-BCT-MM-IM
|
04
|
Production / Shop Floor Control
|
OLI4BW
|
BW-BCT-PP-LIS
|
05
|
Quality Management
|
OLIQBW
|
BW-BCT-QM
|
06
|
Invoice Verification
|
OLI6BW
|
BW-BCT-MM-IV
|
08
|
Transport /Shipment
|
VTBW
|
BW-BCT-LE-TRA
|
08
|
Shipment Costs
|
VIFBW
|
BW-BCT-LE-TRA
|
11
|
SD Sales Orders
|
OLI7BW
|
BW-BCT-SD-SLS
|
12
|
LE Deliveries / Shipping
|
OLI8BW
|
BW-BCT-LE-SHP
|
13
|
SD Billing Documents
|
OLI9BW
|
BW-BCT-SD
|
17
|
Plant Maintenance
|
OLIIBW
|
BW-BCT-PM
|
18
|
Customer Service
|
OLISBW
|
BW-BCT-CS
|
40
|
Retail / Retailing
|
ORISBW
|
BW-BCT-LO
|
45
|
Agency Business
|
OLIABW
|
BW-BCT-ISR-AB
|
IMG path:
- Data Transfer to the SAP Business Information Warehouse
- Settings for Application-Specific DataSources (PI)
- Logistics
- Managing Extract Structures
- Initialization
- Filling the Setup Tables
>> Application-Specific Setup of Statistical data
3- Specify a Run Name and Date (inform a future date).
4- Execute.
Hint: You can check the logs for the Setup in Transaction NPRT.
How can you check the data that was filled?
You may check the correspondent Setup Table or perform an extraction check for the relevant extractors in RSA3.
Initialization or Full Loads
- Delete the Setup Tables contents from the relevant application number.
- Lock all users and terminate all scheduled jobs that could affect the application. In particular, you must deactivate the start of the RMBWV3xx update report.
- Then load the delta data from R/3 into BW, so that the delta queue (transaction RSA7) no longer contains any data.
- Fill the Setup Tables.
- If the datasource supports early delta initialization (ROOSOURCE-ZDDABLE field), you may unlock the users and return regular activities.
- Run the InfoPackage.
- If the datasource doesn't support early delta initialization (ROOSOURCE-ZDDABLE field), you'll be able unlock the users and return regular activities only after the Init request has finished in the BW system.
More details on SAP's recommedations can be found on the following SAP Note:
602260 - Procedure for reconstructing data for BW.
The performance on the construction of the Setup Tables can be improved, with the following SAP Note:
436393 - Performance improvement for filling the setup tables.
What happens if you don't fill the Setup Tables before performing a full or delta initialization?
If you have deleted the content or has never filled them before: No data is extracted.
If you have already filled it and haven't deleted the content: the data extracted won't be up to date.
What happens if you don't delete the Setup Tables before performing a full or delta initialization?
If you don't refill them: the data extracted won't be up to date.
If you refill them: there is a risk you will have duplicate records.
Extractors Maintenance
- Lock users out of R/3.
- Deletion of the contents of Setup Tables for specific applications in transaction LBWG.
- Run the collective run, so all the data is sent into the delta queues.
- Empty BW delta queues on R/3 by extraction to BW.
- Apply the changes / Transport.
- Unlock users out of R/3.
- Run the collective run so all the data are sent into the delta queue.
- Recommence extraction into BW from the modified R/3.
The following SAP Note contains more information about this:
328181 - Changes to extraction structures in Customizing Cockpit.
What happens if you maintain the extractors with data in the Setup Tables or in the Delta Queues?
When you apply the changes (via transport, for example), there is a chance inconsistencies will be generated in the structure of the extractor. The job that collects data for deltas, for example, may start to fail.
It may also appear that delta records were lost. This records are actually written into the extraction queue or in SM13, however it's not possible to read them, because of the differences in the structure.
Note that even if you are adding a new extractor that didn't exist before, this may also affect the existing extractors.
If inconsistencies are found or records are missed, the solution is to perform a Full Repair load or to reinitialize the DataSource. For some applications, you may use the Hash Solution to recover them. The following SAP Note describes this procedure:
835466 - Using the repair mode of the hash solution.
(I'll make sure I do post new post just about the Hash Solution)
In case of missing records, you may also run a Full Infopackage in Repair Mode. You should be able to identify, though, which documents need to me contained in this repair.
R/3 Upgrades
In upgrades, a check was introduced to make sure this procedure is followed. SAP Support's recommendation is to follow the steps listed below, when performing an upgrade.
- Lock users out of R/3.
- Deletion of the Setup Tables contents for specific applications in transaction LBWG.
- Run the collective run so all the data are sent into the delta queue.
- Empty the BW delta queues on R/3 by extraction to BW:
- Start prepare phase on R/3 including generation of shadow instance.
- Return R/3 productive instance to users.
- Recommence normal extraction from R/3 to BW, while upgrading shadow instance.
- Lock users out of R/3.
- Run the collective run so all the data is sent into the delta queue.
- Empty BW delta queues on R/3 by extraction to BW.
- Final upgrade of productive instance.
- Return upgraded system to users.
- Re-commence extraction into BW from upgraded R/3.
SAP Note 1081287 contains a Step-by-Step explanation about cleaning up the Setup Tables and the Delta queues during an upgrade.
1081287 - Data extraction orders block the upgrade process.
As you may be wondering, the act of locking the users in step 1 and 8 isn't the automatic lock the upgrades performs. You will need to lock the users yourself, with transaction SU10, for example. The main purpose is to prevent that data is sent to the extraction queues or Setup Tables.
What happens if you don't properly clean up the Setup Tables in an upgrade?
The consequences are basically the same as changing an extractor: inconsistencies and missing delta records. In this situation, the Hash-Solution isn't really a good option, because the upgrade may have delievered changes on way too many structures. The risk of generating more inconsistencies with the Hash-Solution is great.
Plug-in upgrade or plug-in patch upgrade, support package upgrade or Unicode conversion
SAP Support's recommended Step-by-Step for these activities is very similar to maintaining an extractor.
- Lock users out of R/3.
- Deletion of the contents of Setup Tables for specific applications in transaction LBWG.
- Run the collective run so all the data are sent into the delta queue.
- Empty BW delta queues on R/3 by extraction to BW.
- Apply one of these processes: plug-in upgrade or plug-in patch upgrade; support package upgrade; transports; manual changes; Unicode conversion.
- Unlock users out of R/3.
- Run the collective run so all the data are sent into the delta queue.
- Recommence extraction into BW from modified R/3.
Requirement
may come up to add new fields to LO cockpit extractor which is up &
running in production environment. This means the extractor is
delivering daily deltas from SAP R/3 to BW system .Since this change is
to be done in R/3 Production system, there is always a risk that daily
deltas of LO cockpit extractor would get disturbed. If the delta
mechanism is disturbed (delta queue is broken) then there no another way
than doing re-initialization for that extractor. However this re-init
is not easy in terms of time & resource. Moreover no organization
would be willing to provide that much downtime for live reporting based
on that extractor.
As all of us know that initialization of LO Extractor is critical, resource intensive & time consuming task. Pre-requisites to perform fill setup tables are - we need to lock the users from transactional updates in R/3 system, Stop all batch jobs that update the base tables of the extractor. Then we need to schedule the setup jobs with suitable date ranges/document number ranges.
We also came across such scenario where there was a requirement to add 3 new fields to existing LO cockpit extractor 2LIS_12_VCITM. Initialization was done for this extractor 1 year back and the data volume was high.We adopted step by step approach to minimize the risk of delta queue getting broken /disturbed. Hope this step by step procedure will help all of us who have to work out similar scenarios.
Step by Step Procedure:-
1.Carry out changes in LO Cockpit extractor in SAP R/3 Dev system.As per the requirement add new fields to Extractor.These new fields might be present in standard supporting structures that you get when you execute "Maintain Data source" for extractor in LBWE. If all required fields are present in supporting structure mentioned above then just add these fields using arrow buttons provided and there is no need to write user exit code to populate these new fields.However if these fields (or some of the required fields) are not present in supporting structures then you have to go for append structure and user exit code. The coding in user exit is required to populate the newly added fields. You have to write ABAP code in User exit under CMOD & in Include ZXRSAU01.All above changes will ask you for transport request. Assign appropriate development class/Package and assign all these objects into a transport request.
2.Carry out changes in BW Dev system for objects related to this change.Carry out all necessary changes in BW Dev system for objects related to this change (Info source, transfer rules, ODS, Info cubes, Queries & workbooks). Assign appropriate development class/Package and assign all these objects into a transport request
3.Test the changes in QA system.Test the new changes in SAP R/3 and BW QA systems. Make necessary changes (if needed) and include them in follow-up transports.
4.Stop V3 batch jobs for this extractor.V3 batch jobs for this extractor are scheduled to run periodically (hourly/daily etc) Ask R/3 System Administrator to put on hold/cancel this job schedule.
5.Lock out users, batch jobs on R/3 side & stop Process chain schedule on BW.In order to avoid the changes in database tables for this extractor and hence possible risk of loss of data, ask R/3 System Administrator to lock out the users. Also batch job schedule need to be put on hold /cancel.Ask System Administrator to clear pending queues for this extractor (if any) in SMQ1/SMQ2. Also pending /error out v3 updates in SM58 should be processed.On BW production system the process chain related to delta Info package for this extractor should be stopped or put on hold.
6.Drain the delta queue to Zero for this extractor.Execute the delta Info package from BW and load the data into ODS & Info cubes. Keep executing delta Info package till you get 0 records with green light for the request on BW side. Also you should get 0 LUW entries in RSA7 for this extractor on R/3 side.
7.Import R/3 transports into R/3 Production system.In this step we import R/3 transport request related to this extractor. This will include user exit code also. Please ensure that there is no syntax error in include ZXRSAU01 and it is active. Also ensure that objects such as append structure is active after transport.
8.Replicate the data source in BW system.On BW production system, replicate the extractor (data source).
9.Import BW transport into BW Production system.In this step we import BW transport related to this change into BW Production system.
10.Run program to activate transfer rulesExecute program RS_TRANSTRU_ACTIVATE_ALL. Enter the Info source and source system name and execute. This will make sure that transfer rules for this Info source are active
11.Execute V3 job Manually in R/3 sideGo to LBWE and click on Job Control for Application area related to this extractor (for 2LIS_12_VCITM it is application
12). Execute the job immediately and it should finish with no errors.
12.Execute delta Info package from BW systemRun delta Info package from BW system. Since there is no data update, this extraction request should be green with zero records (successful delta extraction)
13.Restore the schedule on R/3 & BW systemsAsk System Administrator to resume V3 update job schedule, batch job schedule and unlock the users. On BW side, restore the process chains schedule.From next day onwards (or as per frequency), you should be able to receive the delta for this extractor with data also populated for new fields.
As all of us know that initialization of LO Extractor is critical, resource intensive & time consuming task. Pre-requisites to perform fill setup tables are - we need to lock the users from transactional updates in R/3 system, Stop all batch jobs that update the base tables of the extractor. Then we need to schedule the setup jobs with suitable date ranges/document number ranges.
We also came across such scenario where there was a requirement to add 3 new fields to existing LO cockpit extractor 2LIS_12_VCITM. Initialization was done for this extractor 1 year back and the data volume was high.We adopted step by step approach to minimize the risk of delta queue getting broken /disturbed. Hope this step by step procedure will help all of us who have to work out similar scenarios.
Step by Step Procedure:-
1.Carry out changes in LO Cockpit extractor in SAP R/3 Dev system.As per the requirement add new fields to Extractor.These new fields might be present in standard supporting structures that you get when you execute "Maintain Data source" for extractor in LBWE. If all required fields are present in supporting structure mentioned above then just add these fields using arrow buttons provided and there is no need to write user exit code to populate these new fields.However if these fields (or some of the required fields) are not present in supporting structures then you have to go for append structure and user exit code. The coding in user exit is required to populate the newly added fields. You have to write ABAP code in User exit under CMOD & in Include ZXRSAU01.All above changes will ask you for transport request. Assign appropriate development class/Package and assign all these objects into a transport request.
2.Carry out changes in BW Dev system for objects related to this change.Carry out all necessary changes in BW Dev system for objects related to this change (Info source, transfer rules, ODS, Info cubes, Queries & workbooks). Assign appropriate development class/Package and assign all these objects into a transport request
3.Test the changes in QA system.Test the new changes in SAP R/3 and BW QA systems. Make necessary changes (if needed) and include them in follow-up transports.
4.Stop V3 batch jobs for this extractor.V3 batch jobs for this extractor are scheduled to run periodically (hourly/daily etc) Ask R/3 System Administrator to put on hold/cancel this job schedule.
5.Lock out users, batch jobs on R/3 side & stop Process chain schedule on BW.In order to avoid the changes in database tables for this extractor and hence possible risk of loss of data, ask R/3 System Administrator to lock out the users. Also batch job schedule need to be put on hold /cancel.Ask System Administrator to clear pending queues for this extractor (if any) in SMQ1/SMQ2. Also pending /error out v3 updates in SM58 should be processed.On BW production system the process chain related to delta Info package for this extractor should be stopped or put on hold.
6.Drain the delta queue to Zero for this extractor.Execute the delta Info package from BW and load the data into ODS & Info cubes. Keep executing delta Info package till you get 0 records with green light for the request on BW side. Also you should get 0 LUW entries in RSA7 for this extractor on R/3 side.
7.Import R/3 transports into R/3 Production system.In this step we import R/3 transport request related to this extractor. This will include user exit code also. Please ensure that there is no syntax error in include ZXRSAU01 and it is active. Also ensure that objects such as append structure is active after transport.
8.Replicate the data source in BW system.On BW production system, replicate the extractor (data source).
9.Import BW transport into BW Production system.In this step we import BW transport related to this change into BW Production system.
10.Run program to activate transfer rulesExecute program RS_TRANSTRU_ACTIVATE_ALL. Enter the Info source and source system name and execute. This will make sure that transfer rules for this Info source are active
11.Execute V3 job Manually in R/3 sideGo to LBWE and click on Job Control for Application area related to this extractor (for 2LIS_12_VCITM it is application
12). Execute the job immediately and it should finish with no errors.
12.Execute delta Info package from BW systemRun delta Info package from BW system. Since there is no data update, this extraction request should be green with zero records (successful delta extraction)
13.Restore the schedule on R/3 & BW systemsAsk System Administrator to resume V3 update job schedule, batch job schedule and unlock the users. On BW side, restore the process chains schedule.From next day onwards (or as per frequency), you should be able to receive the delta for this extractor with data also populated for new fields.
Hi.. Really like your blog.. somehow cant open given links..please look into it.. looking forward to read more on the blog.. Thanks
ReplyDeletenice post..Retail Software Solution Chennai
ReplyDeleteRetail Software Companies in Chennai
ERP Solution Providers in Chennai
point of sale retail software in chennai
point of sale software in chennai
retail point of sale software in chennai
Hello Dear,
ReplyDeleteI Like Your Blog Very Much. I see Daily Your Blog, is A Very Useful For me.
You can also Find Testbericht Milf-Friend.comMilf-Friend bietet seinen Nutzern nicht nur erotische Unterhaltungen und heiße Flirts, sondern wirbt mit einem direkten sexuellen Kontaktieren und Daten. Inwiefern, diese Voraussetzung tatsächlich umgesetzt wird, steht leider außer unseren Ermittlungen, die wir durchführen konnten
Please Visit at: https://datingsieger.de/testbericht-milf-friend-com/
Blog is very useful but the links are not redirecting to tha valid pages please check and help. It would be great! if those link are accessible to the valid pages.
ReplyDeleteSmm Panel
ReplyDeleteSmm Panel
İş ilanları
instagram takipçi satın al
hirdavatciburada.com
www.beyazesyateknikservisi.com.tr
servis
TİKTOK JETON HİLESİ İNDİR
afyon
ReplyDeletebahçelievler
bingöl
karşıyaka
tekirdağ
C2WS
kırıkkale
ReplyDeletekütahya
niğde
ardahan
bolu
V7EUQ
yurtdışı kargo
ReplyDeleteresimli magnet
instagram takipçi satın al
yurtdışı kargo
sms onay
dijital kartvizit
dijital kartvizit
https://nobetci-eczane.org/
KOAN