Wednesday, April 4, 2012

Data Extraction from CRM to BW with BDOCs

This blog focuses on the concept of data extraction from SAP CRM 7.0 as an OLTP database to SAP BW as a OLAP database. CRM BW Analytics offered by SAP is an extensive and tight integration between the two SAP Business applications. Because the extraction of data works in a very specific way in CRM, it is easily overlooked. Other BW extractions of SAP OLTP work in a more generic way, different from how it is done in SAP CRM 7.0.
There are quite a lot of articles that describe in detail how it is done, this blog gathers that information. By describing it in a simple way it may be easier to understand the basic steps involved in configuring and implementing this CRM BW Adapter framework. The BW adapter (BWA) is built within the CRM environment so that unlike in ECC, no additional installation of a plug-in is necessary.

In a future blog about Data Extraction for CRM, I will explain “How to enhance a BW adapter based data source”.

CRM Analysis

The analysis of information captured by the CRM application can be a good basis for marketing campaigns, opportunity and sales planning and service execution, which is often described as the closed loop scenario by SAP. A closed loop scenario is the cycle from capturing the data, analysis, and applying the results in the operational areas. The results are returned from the operational level to give feedback. A successful CRM solution calls for close integration with a customer information platform, where all relevant customer information is gathered consistently, and analytical insights have to be provided in the operational processes to enable accurate decision making.

When this CRM data is in BW you are able to do the data Mining, CRM Analytics:

  • Clustering; for customer segmentation.
  • Association analysis: for cross selling behaviour
  • Decision trees: for identifying specific profiles that impact a particular behaviour observed Scoring: for combining different aspects in an overall evaluation.
  • Linear and multi-linear regression: to extrapolate data for prediction and behavioural trends.
The constant changing nature of the market and a "Dynamic Customer Need", require technology and application to be based on a framework that can deliver the flexibility it needs.

BDocs (Business Documents)

The CRM BW adapter forms the framework which is primarily responsible for extraction of data into the SAP Business Information Warehouse from the SAP CRM application database. The SAP CRM transactions use BDoc's for writing the transactions to the CRM application database. Every business object, which transacts with the application database to request or to write data, uses a BDoc and each of the CRM business transactions have their own dedicated BDoc. It is like a semantic layer that resides above the CRM application database. A BDoc is defined as a set of transaction statements, which represent a logical object. It encapsulates all business data that are necessary to run a business process. A Business Document has different sets of segments. Segments are mapped to a table where the mapping defines the relation between abstraction and physical database. Segments correspond to logical tables. The fields of a segment are equivalent to the columns of a table.

Data Extraction

Service API is the standard interface in SAP BW environment for source systems. Data extraction from a supply system with help of the Service API is called generic extraction. In SAP CRM, texts and transactional data are normally extracted utilizing the usual performance of the generic BW Service API. With the addition of time-dependent master data and transactional information, the BW Adapter (BWA) in CRM provides a delta mechanism, extracting information from BDocs. CRM-specific datasources are used for this purpose. The delta process must be initialized with a preliminary upload of all enterprise knowledge for a particular CRM object from the CRM database. BW Adapter then displays changes to current objects with assist of the CRM Middleware, and passes them to the BW Service API and the so referred to as BW Delta Queue to the BW system. Knowledge transfer from CRM to BW is carried out with the qRFC expertise for the delta upload.

The technical description

The technical description of the CRM extraction process for both Full and Delta extraction.

Full Data load:

  1. SAP BW calls the BW Adapter using the Service API.
  2. BW Adapter reads the data from the source table for the DataSource.
  3. Selected data is converted into the extract structure as a BDoc.
  4. The BDoc type determine the BAdI that is called by the BW Adapter.
  5. Data Package is transferred to SAP BW using the Service API.
Delta load:
  1. Delta changes in CRM are communicated via Bdoc.
  2. The flow controller for BDocs calls the BW Adapter. 
  3. BW Adapter checks if the delta change is in BDoc that's BW relevant. 
  4. Non-relevant net changes are not sent to SAP BW.
  5. Relevant net changes are transferred to SAP BW.
  6. CRM standard DataSources use AIMD delta method.

Configuration steps for Extracting data from CRM to BW

In the CRM System 

IMG CRM -> CRM Analytics
  • Assign Dialog RFC destination for SAP NetWeaver BI
  • If your default RFC destination is not a dialog RFC destination, you need to create an additional dialog RFC destination and then assign it to your default RFC destination
Transaction SBIW -> Business Content DataSources -> Transfer Application Component Hierarchy
  • Select Application Component hierarchy is transferred
Transaction SBIW ->  Settings for Application specific Data Source (CRM) -> Settings for BW adapter -> Activate BW Adapter Metadata 
  • Select the relevant data sources for CRM
In the BW system
Transaction RSA1 (DW Workbench)
  • Create a source system to establish connectivity with CRM Server. (CRMCLNTxxx)
  • Both BW and CRM should have defined Back ground, RFC users and logical systems)
  • Activate needed Business content of CRM
  • Replicate datasources
In the CRM System
Transaction RSA3
  • Choose data source, to check data which will come over to BW
In the BW system
  • Schedule Initial data load
  • Schedule Delta