Thursday, April 27, 2017

Integrating Salesforce.com over REST API using IBM Integration Bus

Integrating Salesforce.com over REST API using IBM Integration Bus
Introduction
Salesforce.com (abbreviated as SFDC) is the world's leading Software-as-a-Service (SaaS) application that primarily deals in offering CRM software to its consumers. SFDC has contributed immensely to this era of cloud adoption on the applications front. It has seen incredible growth in recent times and has over 100K (and counting) customers already.
IBM Integration Bus (abbreviated as IIB) is IBM's offering in the ESB/middleware space. It was formerly known as Websphere Message Broker. Until recently, the middleware platforms were needed primarily for integrating on-premise applications, especially in a complex application landscape and as the foundation for enabling SOA in a more architecturally (all three disciplines: business, application and infrastructure) mature organizations.
These days, middleware is also needed for integrating on-premise apps with cloud apps in a hybrid IT landscape.
Coming back to SFDC for a moment, even though to the business consumer, SFDC is a business app but its core functionality and power in reality resides in its underlying platform on which the app itself is hosted.
This platform, knows as Force.com exposes a number of means for the consuming apps (middleware included) to integrate with the application, this article focuses on one of them, the REST API.
This article is divided in the following two sections:
  • The SFDC Setup
  • Consuming SFDC via IIB

The SFDC Setup

Before moving to the IIB side, SFDC needs to be configured appropriately for the platform to handle REST based requests.
  • Connected App: Offers a mechanism to integrate the external client with Salesforce using the API and helps authenticate the client using OAuth2.0:
    • To create the connected app, follow these steps:
      • Setup -> Build -> Create -> Apps -> Connected Apps -> New (Button)??
  • Please provide the following information in the given fields (bare minimum per the scope of this article)
    • Connected App Name: Test Connected App
    • API Name: Test_Connected_App (this is the technical name of the app that can be referred inside the given client app, this gets defaulted to the same name as the Connected App Name, minus the spaces)
    • Contact Email: This is the email that SFDC uses to contact you or your support team
    • Under the API (Enable OAuth Settings) section:
      • Select Enable OAuth Settings
      • Callback URL: This field contains that URL of the app to which Salesforce can callback on. Typically, this is the URL that the user's browser gets redirected to after successful authentication. This field is not needed in our case and therefore I am putting in a test/dummy value in this mandatory field
      • Select Access and manage your data (api) option from the "Available OAuth Scopes" selection box
      • Hit Save at the bottom of this page
      • Following is a screen-grab of the create connected app page, for reference:
  • Your connected app is now created. Please give it 2-10 mins to take affect. In the meanwhile, please make a note of the following two parameters that are now available on this newly built connected app.
    • Consumer Key
    • Consumer Secret
  • These params are needed for the client authentication at runtime, in-fact the following 5 params are needed for a successful login attempt using the Username-password OAuth flow:
    • Username: Your Salesforce User ID
    • Password: Your Salesforce account password + User Token associated with your account (this in-fact is a concatenation of the two values)
    • Consumer Key: This will be supplied as the client_id param in the http request for login
    • Consumer Secret: This will be supplied as client_secret param in the http request for login
    • The string value password
We are now done with the SFDC ground work preparation, now moving off to the IIB part:

Consuming SFDC via IIB

For the benefit of the readers of this article, I developed a sample application in IIB to demonstrate the integration part with SFDC.
The application performs the following functions:
  • Receives a request message in the input Q
  • Logs into SFDC using Username-Password OAuth pattern
  • Creates an account in SFDC based on the contents of the request message received in the input Q
  • In case of a successful account creation, put the ID of the created record and the status of the request (both fields are returned by SFDC) in an outgoing XML message that is put in the output Q
Here is the XML request:
<SFDC_Create_Source>
    <Account>
        <AccountName>Test Account 1</AccountName>
        <AccountNumber>123</AccountNumber>
        <Type>Prospect</Type>
    </Account>
</SFDC_Create_Source>
You should be able to find this in the project file attached. Here is the screenshot for reference.
The elements in this xml Account/AccountName, Account/AccountNumber and Account/Type are the params that we will be supplying in our create message before firing it off to SFDC so that an Account record is created there with the provided information.
The message flow looks like this:

Design Time Settings

Following four user defined properties are used in this message flow:
  • UserName: Contains the SFDC account username
  • Password: Concatenated value, combining the password and the security token associated with the given account
  • ClientSecret: Consumer secret, taken off the connected app that was developed earlier in SFDC
  • ClientID: Consumer key, taken off the connected app that was developed earlier in SFDC
In the Fire SFDC login http node,this URL is specified: https://login.salesforce.com/services/oauth2/token (this is the URL on which login needs to be attempted)
For both http nodes: Fire SFDC login and Fire Account Create, set HTTP Method to POST
For both http nodes: Fire SFDC login and Fire Account Create, set the SSL protocol to TLS.
For both http nodes: Fire SFDC login and Fire Account Create, set Message domian to JSON: For JavaScript Object Notation messages in the Response Message Parsing tab
Set Compute mode to All (or at least LocalEnvironment and Message) for Process Login and Prepare Create Command Compute node

The Code

Initialization and login preparation

CREATE COMPUTE MODULE Init_And_Prepare_Login
    DECLARE UserName EXTERNAL CHAR;
    DECLARE Password EXTERNAL CHAR;
    DECLARE ClientSecret EXTERNAL CHAR;
    DECLARE ClientID EXTERNAL CHAR;
    CREATE FUNCTION Main() RETURNS BOOLEAN
    BEGIN    
        --Init, preserving the contents of the source message in 
        --Environment tree using the utlity methods defined above
        -- These contents are needed at different steps in the downstream processing
        -- The reason why these special copy functions are used is 
        -- because the header trees (mqmd, mqrfh2 etc.) cannot be simply assigned
        -- to the Environment root only to be reassigned 
        -- to any other tree in a given compute node. 
        -- The reason is that these headers use
        -- special parsers [DOMAIN keyword with the parser name, 
        -- e.g. DOMAIN(MQMD) or DOMAIN(MQRFH2) is used while 'creating' parsers and
        -- then the fields inside them
        -- More info on this topic can be found in this post:
        -- https://villein.wordpress.com/copyfolders-by-esql-reference/
        
        DECLARE rInput REFERENCE TO InputRoot;
        DECLARE rEnvironment REFERENCE TO Environment;
            --The following methods copies the source message tree to the environment root. 
            --The contents are needed in
            --the subsequent processing steps
        CALL CopyAllFolders(rInput,rEnvironment,'TOP');
    
        --Now building the http post request for loging into SFDC. 
        --Please keep in mind that we are using Username-Password OAuth
        --Authentication Flow in this case
        DECLARE _param1 CHAR ''; DECLARE _param2 CHAR ''; _
        DECLARE _param3 CHAR ''; DECLARE _param4 CHAR ''; DECLARE _param5 CHAR '';
        DECLARE _totalString CHAR '';
        SET _param1 = 'grant_type=password';
        SET _param2 = '&client_id=' || ClientID;
        SET _param3 = '&client_secret=' || ClientSecret;
        SET _param4 = '&username=' || UserName;
        SET _param5 = '&password=' || Password;
        
        --Combining all the above params
        SET _totalString = _param1 || _param2 || _param3 || _param4 || _param5;
        --Setting the URL
        --SET OutputLocalEnvironment.Destination.HTTP.RequestURL = 
        --'https://login.salesforce.com/services/oauth2/token';
        --Setting the Content Type
        SET OutputRoot.HTTPRequestHeader."Content-Type" = 'application/x-www-form-urlencoded';
        SET OutputRoot.BLOB.BLOB = CAST(_totalString AS BLOB CCSID InputRoot.MQMD.CodedCharSetId);
        -- The command should assume this form:
        /*
        grant_type=password&client_id=3MVG9lKcPoNINVBIPJjdw1J9LLM82Hn
        FVVX19KY1uA5mu0QqEWhqKpoW3svG3XHrXDiCQjK1mdgAvhCscA9GE&client_secret=
        1955279925675241571&username=testuser%40salesforce.com&password=mypassword
        */
        -- CALL CopyMessageHeaders();
        -- CALL CopyEntireMessage();
        RETURN TRUE;
    END;

    CREATE PROCEDURE CopyMessageHeaders() BEGIN
        DECLARE I INTEGER 1;
        DECLARE J INTEGER;
        SET J = CARDINALITY(InputRoot.*[]);
        WHILE I < J DO
            SET OutputRoot.*[I] = InputRoot.*[I];
            SET I = I + 1;
        END WHILE;
    END;

    CREATE PROCEDURE CopyEntireMessage() BEGIN
        SET OutputRoot = InputRoot;
    END;
END MODULE;

Process login response and prepare the create Account command

CREATE COMPUTE MODULE Process_login_and_prepare_create_command
    CREATE FUNCTION Main() RETURNS BOOLEAN
    BEGIN
        
        DECLARE _objectURI CHAR InputRoot.JSON.Data.instance_url; --URL of the instance 
                                                 --associated with the given SFDC account
        DECLARE _oauthToken CHAR InputRoot.JSON.Data.access_token; --access token to unlock 
                                                                   --the associated instance
        SET _objectURI = _objectURI || '/services/data/v30.0/sobjects/Account/'; -- setting the 
                                                       --API version and the object on which the
        --insert/create operation needs to be performed, to the URL
        
        -- Setting HTTP params dynamically
        SET OutputRoot.HTTPRequestHeader."Content-Type" = 'application/json';
        SET OutputRoot.HTTPRequestHeader."Authorization" = 'Bearer ' || _oauthToken;
        SET OutputLocalEnvironment.Destination.HTTP.RequestURL = _objectURI;
        
        SET OutputRoot.JSON.Data.Name = Environment.XMLNSC.SFDC_Create_Source.Account.AccountName;
        SET OutputRoot.JSON.Data.AccountNumber = _
            Environment.XMLNSC.SFDC_Create_Source.Account.AccountNumber;
        SET OutputRoot.JSON.Data.Type = Environment.XMLNSC.SFDC_Create_Source.Account.Type;
        
        
        -- CALL CopyMessageHeaders();
        -- CALL CopyEntireMessage();
        RETURN TRUE;
    END;

    CREATE PROCEDURE CopyMessageHeaders() BEGIN
        DECLARE I INTEGER 1;
        DECLARE J INTEGER;
        SET J = CARDINALITY(InputRoot.*[]);
        WHILE I < J DO
            SET OutputRoot.*[I] = InputRoot.*[I];
            SET I = I + 1;
        END WHILE;
    END;

    CREATE PROCEDURE CopyEntireMessage() BEGIN
        SET OutputRoot = InputRoot;
    END;
END MODULE;

Process the create Account response into the outgoing response XML message

CREATE COMPUTE MODULE Process_Create_Result
    CREATE FUNCTION Main() RETURNS BOOLEAN
    BEGIN
        -- Restoring Properties and MQMD from Environment to OutputRoot.
        DECLARE rOutput REFERENCE TO OutputRoot;
        DECLARE rEnvironment REFERENCE TO Environment;
        CALL CopyAllFoldersButLast(rEnvironment,rOutput,'BOT');
        SET OutputRoot.XMLNSC.Response.Id = InputRoot.JSON.Data.id;
        SET OutputRoot.XMLNSC.Response.Status = InputRoot.JSON.Data.success;
        -- CALL CopyMessageHeaders();
        -- CALL CopyEntireMessage();
        RETURN TRUE;
    END;

    CREATE PROCEDURE CopyMessageHeaders() BEGIN
        DECLARE I INTEGER 1;
        DECLARE J INTEGER;
        SET J = CARDINALITY(InputRoot.*[]);
        WHILE I < J DO
            SET OutputRoot.*[I] = InputRoot.*[I];
            SET I = I + 1;
        END WHILE;
    END;

    CREATE PROCEDURE CopyEntireMessage() BEGIN
        SET OutputRoot = InputRoot;
    END;
END MODULE;
There you have it. You can download the project file and go through the solution that way as well. You can also extend this project to fit your requirements.

Important Links

Thursday, March 30, 2017

TRIM Low values X'00

DECLARE StaticValueBlob BLOB CAST('StaticValue' AS BLOB CCSID InputRoot.Properties.CodedCharSetId);

  DECLARE parseOptions INTEGER BITOR(ValidateContentAndValue, ValidateException, ValidateComplete);
        DECLARE vAcctNum CHARACTER;
       
        CREATE LASTCHILD OF Environment.Variables
        DOMAIN 'DFDL';
               
        SET Environment.Variables.DFDL.SUBTREE = InputRoot.DFDL.ROOTRECORD.SUBTREE;
        SET vAcctNum = CAST(ASBITSTREAM(Environment.Variables.DFDL.SUBTREE OPTIONS parseOptions ENCODING InputRoot.MQMD.Encoding  CCSID InputRoot.MQMD.CodedCharSetId TYPE '{}:SUBTREE') AS CHARACTER CCSID InputRoot.MQMD.CodedCharSetId);


copy /Y "C:\Users\govi.reddy\IBM\IIBT10\workspace_dev\BARfiles\MadMoney.bar" "C:\Users\govi.reddy\IBM\IIB\IIB Services\MoneyMovement\Properties\Dev_MoneyMovement.bar"

SELECT ITEM segment.No
        FROM ControlRef.Segments.Segment[] AS segment
        WHERE segment.No = currentSegment;

Working solution:
DECLARE whiteSpace CONSTANT CHARACTER CAST( X'00' AS CHAR CCSID 1208);
TRIM(whiteSpace FROM InRef_RetCode.BE13501_RETURN_CODE)

Thursday, March 9, 2017

SOAP based webservice

1) Create new application.
2)New soap based message model with existing wsdl
3)new message flow
4)drag the wsdl onto the message flow edit on the new flow
5)this gives a skeleton etc. then write business transformation, routing logic's etc.

REST API and JSON sample message



1) New -->start by creating a REST API
2)give name, check radio button import resources and operations  ->next button
3)check radio button select from a file system to get the .json file.

Generating JSON sample message from xml/soap request..

1)copy soap request/JSON message in the the link to generate JSON from xml message http://www.freeformatter.com/xml-to-json-converter.html#ad-output

2)Remove Body{ and at the end closing bracket } from the message and update the first letter of the each field with small letter.

Validating JSON at the below link..

WSDL to Swagger Generator

Generate JSON/YAML swaggers from a WSDL

Installation

  1. Install Node.js >= v6.0.0 (https://nodejs.org/en/)
  2. Clone the repo from SAMI-GIT
    git clone https://git.xyz.com/NAWM_Tools/wsdl-to-swagger.git
    cd wsdl-to-swagger
    
  3. Install dependencies
    npm install
    

Usage

This script takes two inputs: 1) input WSDL file, and 2) output file. The output file needs to end with .json or .yml.
Example:
node wsdl-to-swagger.js example.wsdl example.json
Swagger validation: http://editor.swagger.io/

Thursday, February 16, 2017

IBM Integration Bus Tech Reference

Webcast replay: WebSphere Message Broker - Security Administration


Webcast replay: Integrating WebSphere Message Broker V7 with SAP



Webcast: Migrating to IBM Integration Bus V10



Index: IBM Integration Bus and WebSphere Message Broker trace directions



Webcast replay: Debugging top 5 memory problems in IBM Integration Bus (IIB)



Open Mic Replay: Security in WebSphere Message Broker



Webcast replay: Implementing SSL with HTTP nodes in WebSphere Message Broker



Webcast replay: WebSphere Message Broker - Security in SOAP nodes



Webcast Replay: IBM Integration Bus - What's new



Webcast replay: Common Problems While Configuring SSL in v8 Message Broker



Webcast replay: How do you use .Net with the Broker



Webcast replay: Using TCP/IP Based Applications with WebSphere Message Broker



Webcast replay: Events-Based Messaging Monitoring in WebSphere Message Broker V7



Webcast replay: Using WebSphere Message Broker Configurable Services



Webcast replay: Packaging and Deploying Java Based Solutions to WebSphere Message Broker


Webcast Replay: Tools and techniques for performance testing IBM Integration Bus (IIB) solutions


This procedure is typically run to track down problems in IBM Integration Bus (IIB) or Websphere Message Broker (WMB) with database connectivity such as user access and dropped database connections. Gathering this information before calling IBM Support will help familiarize you with the troubleshooting process and save you time.


The following is a list of answers to frequently asked questions (FAQ) about Migration in IBM Integration Bus (IIB) and WebSphere Message Broker (WMB) for new and experienced users.


The following is a list of answers to frequently asked questions (FAQ) about Database in IBM Integration Bus (IIB) and WebSphere Message Broker (WMB) for new and experienced users.


Performance is degraded when IBM Integration Bus (IIB) or WebSphere Message Broker (WMB) calls stored procedures.


Use this procedure to capture a user or service level trace of an IBM Integration Bus (IIB) message flow at an Integration Server level or a WebSphere Message Broker (WMB) message flow at an execution group level. 

Gathering this information before calling IBM Support will help familiarize you with the troubleshooting process and save you time.


This Index is a compilation of links to documents that provide instructions for gathering various trace data on IBM Integration Bus (IIB) and WebSphere Message Broker (WMB) and related products.
Content
Trace can gather additional details required for problem solving. Follow the directions provided in these documents to gather trace details.


Use this procedure to capture a service level trace of the "Admin agent" (bipbroker process) in Websphere Message Broker or IBM Integration Bus. 

Gathering this information before calling IBM Support will help familiarize you with the troubleshooting process and save you time.


Use the procedures below to take a startup trace of a broker or execution group in WebSphere Message Broker (WMB). 

IBM Integration Bus (IIB) V9 replaces Execution Group with Integration Server. Similar instructions are provided below for collecting Integration Server startup trace. 

Gathering this information before calling IBM Support will help familiarize you with the troubleshooting process and save you time.


Use this procedure to capture a IBM Integration Bus (IIB) or WebSphere Message Broker (WMB) trace of a failing deploy to a broker. 

Gathering this information before calling IBM support will help familiarize you with the troubleshooting process and save you time.

Integration Server or Execution Group memory growing
Your IBM Integration Bus (IIB) Integration Servers or WebSphere Message Broker (WMB) Execution Groups appear to be growing in memory indefinitely.

Increasing the maximum JVM heap size of the WebSphere Message Broker HTTP listener

The broker-wide listener process (biphttplistener) handles both inbound requests and outbound replies for the HTTPInput and HTTPReply nodes across an entire broker. By default, this listener is started with a maximum JVM heap size of 192MB. 

It is possible, based on inbound request rates and message sizes, that the default maximum size is not sufficient.



Configuring and administering multi-instance brokers for high availability in IBM WebSphere Message Broker - Part 1
This article describes multi-instance brokers, a feature introduced in WebSphere Message Broker V7 that helps you configure high-availability environments. The feature is based on multi-instance queue managers in WebSphere MQ V7.0.1, which provide multiple instances of a queue manager residing on separate physical servers to provide a highly available configuration. In WebSphere MQ, the multi-instance queue manager's configuration is hosted on shared network storage, so that if the active queue manager fails, another instance automatically assumes the stopped queue manager's configuration and becomes active. Similarly, in Message Broker V7 and V8, a broker's configuration is hosted on network storage to enable the high-availability function.

Configuring and administering multi-instance brokers for high availability in IBM WebSphere Message Broker - Part 2
This article describes the active-active technique for high availability using both vertical and horizontal clustering, which compared to the active-passive technique, improves continuous availability, performance, throughput, and scalability in WebSphere MQ and WebSphere Message Broker.

IBM Datapower Developerworks Articles


IBM Datapower DeveloperWorks Articles
4.    Integrating the WebSphere DataPower SOA Appliance XI50 with the WebSphere Application Server V7 default JMS messaging provider
6.    Using OAuth on IBM WebSphere DataPower Appliances
7.     SOA governance using WebSphere DataPower and WebSphere Service Registry and Repository
8.    Leverage DataPower SOA Appliances to extend InfoSphere Master Data Management Server security capabilities
9.    Using WebSphere DataPower and WebSphere MQ File Transfer Edition to manage file transfers
10.  Configuring a WebSphere DataPower Kerberos-secured backend server
11.   Overriding the WebSphere DataPower WSDL optimization conversion
13.  Using the WebSphere DataPower Option for Application Optimization to demonstrate self-balancing across multiple DataPower appliances and intelligent load distribution to backend servers
14.  Integrating WebSphere DataPower SOA Appliances with WebSphere MQ
15.  Performance advantages of using WebSphere DataPower XC10 Appliance as a side cache for a Java application
16.  Integrating DataPower with WebSphere Message Broker using the Broker Explorer
17.  Integrating WebSphere DataPower XML Security Gateway XS40 with WebSphere Message Broker
20. Integrating Web applications with the DataPower Web application firewall service
21.  Managing WebSphere DataPower SOA Appliance configurations for high availability, consistency, and control, Part 1
22. Managing WebSphere DataPower Device configurations for high availability, consistency, and control, Part 2: Application promotion strategies
24. Using DataPower SOA Appliances to query WebSphere Service Registry and Repository

Distributed Computing: A Guide to Comparing Data Between Hive Tables Using Spark

In big data, efficient data comparison is essential for ensuring data integrity and validating data migrations. Apache Spark, with its in-me...