Understanding Integration Cloud and how to get the most out of your implementation

Tag: FTP

August 2023 – New Additions

Article / LinkAuthorSubject MatterConnecting
Scope In Oracle Integration CloudTechtutorstiOIC 
OIC: Reverse Versioning IntegrationsJan KettenisOIC
How to convert XML data to Json data and store it in a variableSanddyOIC
XSD – How to validate if any field is empty while reading the file | add restriction to a schema fieldSanddyOIC
Oracle Integration Cloud – Role-Based Control Access with ProjectsAna Lucero AlvaradoOIC 
How to delete an Orphan Fusion Apps Subscription created by OICJin ParkOIC 
OCI Toolkit for Visual Studio CodeDaniel TeixeiraAdvanced Protection: OIC generation 2 & File Server 
Oracle Integration Cloud – Role Based Control Access with ProjectsDaniel TeixeiraOIC 
Oracle Integration Cloud – ERP Integration Patterns – Real-Time InboundDaniel TeixeiraOIC 
Provision and configure Oracle Integration Cloud instance using TerraformOmid IzadkhastiOICTerraform
Intelligent Document Processing with Oracle Integration Cloud and OCI AI Vision ServiceJerry Mbamo, Nolan Trouvé, Niall CommiskeyOICAI
OCI Logging Analytics OOTB OIC DashboardsNiall CommiskeyOICLogging ANalytics
SFDC User Sync with OIC and other things SFDC adapterNiall CommiskeyOICSFDC
OIC3 Service MetricsNiall CommiskeyOIC3 
OIC 3 Billing Metrics and AlarmsNiall CommiskeyOIC3 
OIC 23.08 New Features (Object Storage, OIC Events, HCM Adaptor, Project Stats, Adaptors)Niall CommiskeyOIC 
OAuth In Oracle Integration CloudTechtutorstiOIC 
Retry In Oracle Integration CloudTechtutorstiOIC 
Oracle Integration Cloud SFTPTechtutorstiOIC 
New Features in Oracle Integration 3! (23.08)Amy Simpson-GrangeOIC3
Syncing Oracle Content Management Documents from PROD to DR using OICBimandsOIC 
Oracle BICC Work Flow with auto file extraction from BICC UCM Server – OICeoracleappsOICBusiness Intelligence Cloud

April 21 – New OIC articles

Blink and it’s another month gone. But Oracle has been very very busy as we come around for the latest quarterly updates and the details published. Plus a lot about Oracle Hospitality Integration Hub (OHIP) which builds upon OIC.

Article / LinkAuthorSubject MatterConnection
May 2021 Oracle Integration UpdateAntony ReynoldsOIC 
May 2021 Third-Party application adapters updateMichael MeinerAdapters 
May 2021 Oracle Integration Fusion Application Adapters ImprovementsPrakash MasandSaaS Adaptors 
May 2021 Oracle Integration Database Adapter ImprovementsPakash MasandDB Adaptors 
May 2021 Enhanced Security in AdaptersMichael MeinerAdapters 
May 2021 Technology Improvements in AdaptersMichael MeinerAdapters 
Connect Oracle SaaS (ERP, HCM, CX) with Pre-Built Recipes – Accenture Oracle Webcast May 5th 2021Amit Tahilramani, Mani Choudhary and Amit Singh with Dhiraj ChhajerOracle SaaS Recipes 
Invoking Process Applications with Anonymous Users from VB Apps with Oracle IntegrationArvind Venugopal, Shay Shmeltzer, and Nathan AngstadtVisual Builder 
Extend Process Automation UI Components to JET-Based ApplicationsArvind Venugopal, Rameshkumar Arumugam, and Sacheth SavandurgaProcess 
Using an Event Based Pattern with Oracle IntegrationMichael MeinerOIC PatternShopify
Practical Guide to use HCM Data Loader with Oracle IntegrationPrakash Masand, and Harris QureshiOICHCM
Using UN/EDIFACT for B2B TransactionsArvind Venugopal, and Sunil ApteOIC 
Modelling Transport Integrations with Trading Partner ManagementArvind Venugopal, and Sunil ApteOICOICTrading Partner
May 2021 Oracle Integration Pre-built RecipesMichael MeinerOICNetSuite, ServiceNow, Facebook, Slack, docusign
Connecting securely from Oracle Integration to Autonomous database using network access listShreenidhi RaghuramOICAutonomous DB
OIC – Oracle Hospitality syncing Company/Guest with SFDC Account/ContactNiall CommiskeyOICOracle Hospitality
OIC May 21 Release – New Jira adapterNiall CommiskeyOICJIRA
OIC for File Transfer – Rename – Move to Target – Delete SourceNiall CommiskeyOICFTP
OIC –> Oracle Hospitality – refreshing OAuth TokenNiall CommiskeyOICOracle Hospitality
Box to HCM Cloud –> HDL File LoadNiall CommiskeyOICBox
Integrating Oracle Hospitality Cloud and 3rd party CRM (SFDC) via OIC and OHIPNiall CommiskeyOICOracle Hospitality
OIC and Oracle Hospitality (OHIP) – Getting the most out of Opera APIsNiall CommiskeyOICOracle Hospitality
Five key Oracle Integration lessons from handling billions of messages monthlyAmit Saxena & Daryl EicherOIC 
Working Around not Having Complex GatewayJan KettenisOIC 

Beating OIC’s FTP File Limits

OIC has for some time now provided an FTP adaptor and more recently included a full FTP server capability. But both have limits on the file size and capacity. The file constraints (1GB for a file and 500GB for the FTP server) shouldn’t be an issue for day to day activities. But OIC is often used to support SaaS Financials and other cloud solutions which do have monthly process cycles which can generate significant data volumes, for example, payroll data. The question is how to handle such data with such constraints?

Data Integration?

There is a school of thought that points to the possibility when handling such large data volumes we should consider using Data Integration rather than a more event-centric integration tool. Personally, I think there is a lot of validity in the argument, and anyone dealing with such bulky data activities should review and question if it is a better answer.

That said, there are cases where it does stand-up. For example:

  • If an organization is transitioning to a more event-driven or at least micro-batch model, you have to start the transition somewhere, but trying to line up changes everywhere can be problematic, so we have to start somewhere. Building an integration process so you have an event model developed, but in the interim, you need to take that bulk mechanism and convert it to a small stream of events.
  • You may be working with a bulk data extract and only need a small subset of the data provided, it won’t help if the data is also represented using a verbose notation such as XML.

Other Approaches

How to overcome the constraint? Oracle databases aren’t so constrained, and SQLLoader can provide an easy means to ingest the data into a staging table. The benefit of this is:

  • if you’re only needing a subset of the data you can pull just those columns from the table.
  • the bulk of the use of XML to be self-describing can be shed through using the DB schema as being more prescriptive.
  • SQL scripts can handle the checksum records removing that data and overhead from the integration process, leaving you to concentrate more on the business process.

If the data is still substantial once in the database there are a number of strategies to consume the data in more manageable chunks, such as

  • Running SQL script on the database that takes each row and calls the OIC as a restful API point. This approach is potentially very interesting as it may then mean if you’re moving towards an event process in the future the API endpoint represents the future state and the database stored procedure is mimicking the future client behaviour.
  • Use polling strategies and result set limits to control how much data is processed in a single execution of the integration. This approach does mean the integration needs to tag which records have been processed to avoid re-reading them.

Useful Links

Public FTP Services that could be used with Chapter 9

In chapter 9 of the book we work through a number of examples of using ICS with FTP.  If you don’t have a web provider that already offers you an FTP server to use you, we’ve found a few services you might consider using. We provide no guarantees for their trustworthiness and reliability. But they maybe sufficient to enable to run the scenarios described in the book:

The alternate option is to exploit the fact that website hosting compaies usually provide FTP access to upload content. Here are a couple of options:

Powered by WordPress & Theme by Anders Norén