Search This Blog

How to read GB file or more than 10 MB file using File Adapter - SOA - BPEL

How to read GB file or more than 10 MB file using File Adapter - SOA - BPEL


Most of the time in SOA suite have a suitation to handle more than 10 MB of data to read. But SOA file adapter limitation only 10MB beyond that adapter will fail to read input data. Solution to read GB input file also using Chunk Read logic .



Create normal file adapter using - Sync Read option and modify the .jca file completely

.JCA File :

Chunk Size I defined 100 record / chunk

<adapter-config name="ReadFileChunk" adapter="File Adapter" wsdlLocation="ReadFileChunk.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

  <connection-factory location="eis/FileAdapter"/>
  <endpoint-interaction portType="SynchReadFile_ptt" operation="synchReadFile">
    <interaction-spec className="oracle.tip.adapter.file.outbound.ChunkedInteractionSpec">
      <property name="DeleteFile" value="true"/>
      <property name="PhysicalDirectory" value="D:\Out"/>
      <property name="FileName" value="Krishna.txt."/>
      <property name="ChunkSize" value="100"/>
     </interaction-spec>
  </endpoint-interaction>
</adapter-config>



BPEL Implementation :


Declare default value before while loop logic


 LineNumber=  "1"
 ColumnNumber= "1"
 IsEOF = "false"


<while name="While_ChunkLoop"
               condition="bpws:getVariableData('VarChunkJcaProperties','/ns7:ChunkFileAdapterProp/ns7:IsEOF') ='false'">
          <sequence name="SeqChunkLogic">
            <invoke name="Invoke_ReadFileFromWorkingDir"
                    inputVariable="ReadFile_InputVariable"
                    outputVariable="ReadFile_OutputVariable"
                    partnerLink="ReadFileChunk" portType="ns8:SynchReadFile_ptt"
                    operation="synchReadFile" bpelx:invokeAsDetail="no">
              <bpelx:inputProperty name="jca.file.FileName"
                                   variable="VarChunkJcaProperties"
                                   query="/ns7:ChunkFileAdapterProp/ns7:file"/>
              <bpelx:inputProperty name="jca.file.Directory"
                                   variable="VarChunkJcaProperties"
                                   query="/ns7:ChunkFileAdapterProp/ns7:dir"/>
              <bpelx:inputProperty name="jca.file.LineNumber"
                                   variable="VarChunkJcaProperties"
                                   query="/ns7:ChunkFileAdapterProp/ns7:LineNumber"/>
              <bpelx:inputProperty name="jca.file.ColumnNumber"
                                   variable="VarChunkJcaProperties"
                                   query="/ns7:ChunkFileAdapterProp/ns7:ColumnNumber"/>
              <bpelx:inputProperty name="jca.file.IsEOF"
                                   variable="VarChunkJcaProperties"
                                   query="/ns7:ChunkFileAdapterProp/ns7:IsEOF"/>
              <bpelx:outputProperty name="jca.file.LineNumber"
                                    variable="VarChunkJcaProperties"
                                    query="/ns7:ChunkFileAdapterProp/ns7:LineNumber"/>
              <bpelx:outputProperty name="jca.file.ColumnNumber"
                                    variable="VarChunkJcaProperties"
                                    query="/ns7:ChunkFileAdapterProp/ns7:ColumnNumber"/>
              <bpelx:outputProperty name="jca.file.IsEOF"
                                    variable="VarChunkJcaProperties"
                                    query="/ns7:ChunkFileAdapterProp/ns7:IsEOF"/>
              <bpelx:outputProperty name="jca.file.IsMessageRejected"
                                    variable="VarChunkJcaProperties"
                                    query="/ns7:ChunkFileAdapterProp/ns7:returnIsMessageRejected"/>
              <bpelx:outputProperty name="jca.file.RejectionReason"
                                    variable="VarChunkJcaProperties"
                                    query="/ns7:ChunkFileAdapterProp/ns7:returnRejectionReason"/>
              <bpelx:outputProperty name="jca.file.NoDataFound"
                                    variable="VarChunkJcaProperties"
                                    query="/ns7:ChunkFileAdapterProp/ns7:returnNoDataFound"/>
            </invoke>

        </while>



Spring Logic IHelper Bean :

             chunkSize = 1000;
             lineNumber = "1";
             columnNumber = "1";
             currentLineCount = 1;
       
            while (currentLineCount <= totalLineCount - 1) {
                currentLineCount = currentLineCount + chunkSize;
             
             
                               headerHelper.setHeaderProperty("jca.file.FileName",propertiesMap.get("IN_DENT_FILENM"));
                headerHelper.setHeaderProperty("jca.file.Directory", filePath);
                headerHelper.setHeaderProperty("jca.file.LineNumber",  lineNumber);
                headerHelper.setHeaderProperty("jca.file.ColumnNumber", columnNumber);
         
                 lineNumber = Integer.toString(Integer.parseInt(lineNumber) + chunkSize);
            }
            

normalize-space() - Be Clear where to use in SOA

normalize-space() - Be Clear where to use in SOA


normalize-space() - Performs remove unwanted spaces in a string . We know that but how it works internally.

Example : "      Hai       User             Weclome     you            "

Input String contains spaces in the front , last and middle also. I want to remove spaces in front,middle and last.

Note : normalize space will remove extra spaces in between words also and put only one character space.

normalize-space("      Hai       User             Weclome     you            ") ;


Ouput :    "Hai User Welcome you"


If you are looking for front and last to remove spaces use 

oraext:right-trim(oreaxt:left-trim("      Hai       User             Weclome     you            "));


Output : "Hai       User             Weclome     you"


xp20:format-dateTime - SOA- Custom format Date Time

xp20:format-dateTime - SOA- Custom format Date Time


Middleware  intreacts with mostly external system - Each system need some custom Date and Time format structure. Most of the time developers not remember Syntax .

Y001 - year
M01 - month
D01 - date
H01 - hour
m01 - minutes
s01   -  seconds
f001 - millsec

Use Ready Made functions:


xp20:format-dateTime(xp20:current-dateTime(),'[Y0001][M01][D01]_[H01][m01][s01]')


Including MillSeconds :

xp20:format-dateTime(xp20:current-dateTime(),'[M01]/[D01]/[Y0001] [H01]:[m01]:[s01].[f001]')

Universal Time format with T :


xp20:format-dateTime(xp20:current-dateTime(),"[Y0001]-[M01]-[D01]T[H01]:[m01]:[s01].[f001]")