Note 821267 - FAQ: XI 3.0 / PI 7.0/ PI 7.1 File Adapter 3

31. Empty File Processing

Q: How does the File Adapter handle empty files and messages with a zero-sized payload?

A: Up to and including XI 3.0 SP18 / PI 7.0 SP9 empty files in the sender adapter will not trigger the creation of a message. If an empty message is sent to a receiver channel, no output file will be created.

Starting with XI 3.0 SP19 / PI 7.0 SP10 the channel's behavior can be configured in Integration Builder.

Q: I configure the receiver channel with File content conversion mode and I set the 'Empty Message Handling' option to ignore. Input payload to the receiver channel is generated out of mapping and it does not have any record sets. However, this payload has a root element. Why does file receiver create empty output file with zero byte size in the target directory? Example of such a payload generated from mapping is as follows: <?xml version="1.0" encoding="UTF-8"?> <ns1:test xmlns:ns1="http://abcd.com/ab"></ns1:test>

A: If the message payload is empty (i.e., zero bytes in size), then File adapter's empty message handling feature does NOT write files into the target directory. On the other hand, if the payload is a valid XML document (as shown in example) that is generated from mapping with just a root element in it, the File Adapter does not treat it as an empty message and accordingly it writes to the target directory. To achieve your objective of not writing files (that have just a single root element) into the target directory, following could be done: - Either modify your mapping so that empty message payload without a root element is generated. This could be done only by creating a Java mapping and implement the logic to convert from root element to an empty text. This can not be done using Message Mapping tool. - or, use a module based approach, e.g., using the StrictXml2PlainBean, where such a valid XML payload is converted to an empty payload before the file receiver channel is invoked.

32. File Construction Mode "Add Counter": Counter Reset

Q: After configuring a File Adapter receiver channel with the File Construction Mode set to "Add Counter", I noticed that the counter gets reset each time I restart the J2EE engine.

A: This behavior is by design. The current counter value is not persisted, but only kept in memory. It is intended to generate distinct file names, but not strictly increasing numbers. After a J2EE restart, the File Adapter will start writing messages beginning with the smallest free counter number (skipping existing larger numbers).

33. FTPS: "AUTH TLS" Command Unrecognized

Q: After configuring a File Adapter channel with the Transport Protocol "FTP" and connection security set to "FTPS", I receive the following error in the adapter monitoring: Syntax error, command unrecognized: 'AUTH TLS'

What is causing this?

A: Your FTP server implementation does not implement FTPS as specified in RFC 4217. For details, contact your FTP server vendor.

34. FTP: Directory Specification / CWD Command

Q: Is there any way to configure a File Adapter channel with the Transport Protocol "FTP" without an explicit source / target directory, so that no CWD command will be issued?

A: The directory specification is mandatory. According RFC 1123 an FTP server is required to support the CWD command.

35. File Sender Error Handling

Q: I have configured a File Sender channel with File Content Conversion or a custom-developed module, which may throw an exception. When the File Adapter encounters an invalid file, which triggers a conversion error or an exception in a module, the adapter will enter a retry interval and upon each following processing attempt try to process the faulty file. This basically prevents the File Adapter from picking up files that are located after the fauilty file according to the configured sort order. How can I change this behavior?

A: This is by design. When the File Adapter encounters an invalid file, manually remove this file from the configured source directory. You can configure an alerting rule to get notified each time this occurs.

We are currently evaluating possible solutions that could be used to skip this manual interaction.

36. FTP: Adapter Polls More Frequently Than Configured

Q: I have configured a File Adapter Sender channel with the transport protocol "FTP". When no file is present in the configured input directory, the adapter connects to the FTP server at least once a second or even more frequently. What is causing this?

A: Most likely, this behavior is caused by an implementation error in the FTP server, which you are connecting to:

The FTP protocol specification (RFC 959) requires that when issuing a NLST command to retrieve a file list for a given wildcard pattern a list of valid file names gets returned. Error messages must be transmitted via the control channel. From the specification:

"NAME LIST (NLST)

This command causes a directory listing to be sent from server to user site. The pathname should specify a directory or other system-specific file group descriptor; a null argument implies the current directory. The server will return a stream of names of files and no other information. The data will be transferred in ASCII or EBCDIC type over the data connection as valid pathname strings separated by <CRLF> or <NL>. (Again the user must ensure that the TYPE is correct.) This command is intended to return information that can be used by a program to further process the files automatically. For example, in the implementation of a "multiple get" function."

However, your FTP server returns a human-readable error message in the data connection instead of a file list when no files match the wildcard pattern (the channel's file name mask) provided to the NLST command, which is - in compliance with RFC 959 - interpreted as a file name by the File Adapter.


 

An example FTP session:

[...]

ftp> ls file*.txt

200 PORT command successful.

150 ASCII data connection for /bin/ls

file*.txt: No such file or directory

226 ASCII Transfer complete.

[...]

The line causing the problems reads:

file*.txt: No such file or directory

As this line is interpreted as a file name by the File Adapter (as per RFC 959), the File Adapter will try to retrieve this file, which will - of course - fail. However, as the FTP server has returned at least one (although invalid) file name, the File Adapter will immediately poll again for new files. This is by design and cannot be configured.

The root cause of the problem is the incorrect reply of your FTP server, so you will have to contact your FTP server vendor for a correction, which provides a RFC 959 compliant implementation of the FTP protocol.

37. FTP: sun.io.MalformedInputException

Q: I am trying to connect the File Adapter to an FTP server, which uses non-ASCII replys over the control connection, e.g. the server replys contain German umlauts or accented characters. However, the File Adapter just displays an error referring to sun.io.MalformedInputException when trying to establish the connection. How can I solve this?

A: Configure the "file.encoding" system property to match the encoding returned by the FTP server. For details, refer to the answer to Q20.

38. Windows (SMB) Shares

Q: Using a File Adapter channel I would like to retrieve files from and/or write files to a Windows (SMB) share. Which considerations do I need to take into account?

A: When the Adapter Engine hosting the File Adapter service is running under Microsoft Windows, make sure that the share is accessible for the user "SAPService<SID>" under which the File Adapter is operating. If the share is mapped to a drive letter, make sure that this mapping exists for the "SAPService<SID>" user. UNC paths are typically supported under Microsoft Windows. When the Adapter Engine hosting the File Adapter service is running under UNIX, make sure that the SMB share is mounted into the local file system hierarchy and accessible to the "<sid>adm" user. Note that UNC paths are typically not supported under UNIX. For details, contact your operating system vendor.

39. FTP Sender: 550 File Not Found

Q: I have configured an FTP sender channel, which however fails to retrieve files during each polling interval. Instead, it gives an error message similar to: "550 File Not Found" (the actual wording depends of the FTP server software, which is not provided by SAP). How can I fix this?

A: Configure an absolute source directory (usually starting with a slash) instead of a relative one.

40. Known Problems and Incompatibilities for FTP Servers

Q: What are the known problems and incompatibilites for third party FTP server implementations ?

A: Please refer to note 1052867.

41. Test FTP Server Connection.

Q: How to Test FTP Server Connection.?

A: Please refer to note 1085539.


 

42. Channel prioritization using OS command

Q: I wish to configure two sender File channels sothat second sender channel processes only after first sender channel has finished sending message to XI using an external OS command . I wish to select quality of service as Exactly Once In order(EOIO) for both channels and wish to configure the same queue name for both channels. This way, I want to see that all the messages from second channel are processed only after first channel. Is this supported ?.

A: This is not supported as each channel execution is independent of other channel. Though this may seem to work for most cases we do not guarentee that "at all times" message order is guarenteed. One of the reasons is, though messages from two channels go into the same queue, they may not arrive at the queue in the order that we expect.

43. J2EE engine hangs with File/FTP sender channels.

Q: J2EE engine hangs with the File/FTP sender channels. How to resolve this ?

A: The reason for this is, some times due to network level issues, a message is waiting forever for a response from a FTP server which is down for some time. Adapter does not know this and try to poll FTP server again with second message. This goes on and on and eventually we would have J2EE engine hanging. To solve this, following things need to be applied:


 

1) Set FTP timeout to appropraite value if channel is FTP sender channel.

2) In advanced mode table options, add a new parameter 'clusterSyncMode' and its value set to 'lock'. This parameter is without single quotes and case-sensitive. Please read SAP note 801926 to know more about this option.

3) Last but not least, make sure that you are in latest patches of SP19/SP20/SP21/SP22 for SAPXIAFC component of XI 3.0 release and SP10/SP11/SP12/SP13/SP14 for SAPXIAFC component of XI 7.0 release. Any patch which is released after 11th of Febraury 2008 for the above releases is fine.


 

44. File Receiver with "append" mode on multiple cluster nodes.

Our landscape has file receiver channels configured to use "append" operation. J2EE engine is running on multiple clustered nodes with mulitple central instances and multiple dialog instances. Central instance is running on one physical host and dialog instance is running on another physical host. All cluster nodes need to access the same file to write data in append mode. Do we support this case ?

A: Detailed answer is given in SAP note 1162687.


 

45. Receiver File Adapter stuck while using temporary file.


 

Q :The thread of receiver file adapter remains in runnable while

Communication channel has the following configuration

File Construction Mode=Create, Overwrite Existing File=false, Write Mode= Use Temporary File.


 

A :The Receiver file adapter thread is stuck in the runnable state when the temporary file is used and over writing of existing file is set to false. Since the temporary file was not being deleted and the overwriting of existing file i.e. target file was set to false, the thread would never end. This problem has been fixed by deleting the temporary file after being named to the target file.

The error is corrected with SAPXIAF06P patch#8 (Support Package -level #06).


 

46. How to manually delete hanging Enqueue-Locks

Q :During the OOM error the J2ee engine goes down while the lock in the enqueue server still persists.When the channel comes back during restart as the effect of OOM error it will not able to get the cluster wide enqueue lock as the previous lock remains there in the enqueue server.How do I resolve this?

A :The existing (hanging) enqueue server locks for the corresponding FILE channels(with FILE Lock Arguments) can be deleted manually by following procedure:


 

1. Stop the corresponding FILE channel.


 

2. Open Visual Administrator.


 

3. Route to correspondig server.


 

4. Expand services and go to Locking adapter.


 

5. In the locking adapter service go to Display locks tab and click on refesh button then you will able to see the list of all the available locks.


 

6. Select the existing (hanging) enqueue lock with FILE Argument from the list.


 

7. Click on "Delete Selected Locks" button.


 


 

47. Effect of setting Maximum Concurrency and poolWaitingTime

Q: What is the meaning of different values of Maximum Concurrency? How should be poolWaitingTime set for different values of Maximum Concurrency?

A: Maximum Concurrency Parameter is used to acquire parallel FTP connections for the same receiver File/FTP channel.

Maximum Concurrency = 1

This is the default value of Maximum Concurrency. This means a single receiver File/FTP channel can acquire only one FTP connection. This case does not offer any parallel processing of messaging by single channel.


 

Maximum Concurrency > 1

This ensures that resource pool size (FTP conneciton size) is limited to specified value per single File/FTP receiver channel. For instance, setting this parameter to 2 will ensure that a single receiver File/FTP channel can acquire 2 FTP connections in parallel(if needed).


 

poolWaitingTime

This timeout value will define the maximum waiting time to get the connection from resource pool. If connection is not available within this time, we return the message to Messaging System and the message is in retry mode.


 

The configuration of this timeout requires the use of an advanced mode configuration parameter.


 

To configure a timeout for the File/FTP receiver channels, please enable the "Advanced Mode" for the respective File/FTP communication channel in the Integration Directory and add an entry "poolWaitingTime" and set to a time in Milli Secs (without any quotation marks) to the "Additional Parameters" section. This parameter is case-sensitive.


 

The default value of this parameter is 5000 Milli Secs.


 

The default value ensures a channel will wait 5000 Milli Secs before aquiring connection.


 

1. Maximum Concurrency = 1


 

When Maximum concurrency is set to 1, poolWaitingTime should be set to 0. This makes sure that the channel can wait infinitely to get the FTP connection. This will prevent messages to get into error state un-necessarily.


 

2. Maximum Concurrency > 1


 

Before setting this, please apply note 1136790. For Maximum Concurrency > 1, this note should be applied and then only proceed with the following.


 

When Maximum Concurrency > 1 poolWaitingTime must be set to an optimal value. This should be more or less equal to time required to aquire a connection on a remote FTP/DB. If this is not known, set this to 1 min(60000 Milli Secs). This will ensure that, if channel cannot get the FTP connection with in the time specified then after 3 retries the thread will be released for other channels and message back-log can be reduced.


 


 


 

Note 821267 - FAQ: XI 3.0 / PI 7.0/ PI 7.1 File Adapter 2

21. File Sender: Performance Issues After Configuring A Large Amount Of Channels

Q: After configuring a large amount of File Adapter sender channels, the J2EE Engine becomes very slow and some services start to block. How can I solve this issue?

A: Up to and including XI 3.0 SP13 each File Adapter sender channel permanently consumes a J2EE application thread. To solve this issue, increase the number of configured J2EE application threads using the SAP J2EE Engine Config Tool ("cluster-data" -> "Global server configuration" -> "managers" -> "ApplicationThreadManager" -> "MaxThreadCount").

Starting with XI 3.0 SP14 / PI 7.0 application threads are allocated on demand by the File Adapter and returned to the thread pool after it has finished the polling sequence, so thread shortage situations will typically occur much more rarely than with earlier SPs.

22. File Sender: File Modification Check For Transport Protocol "FTP"

Q: Is the file modification check (advanced mode option "Msecs to Wait Before Modification Check") in a File Adapter sender channel supported for the transport protocol "File Transfer Protocol (FTP)"?

A: No, this option is not supported for the transport protocol "File Transfer Protocol (FTP)", but only for the transport protocol "File System (NFS)".

23. FTP: Active Data Connection Support

Q: Does the File Adapter support active data connections when connecting to an FTP server?

A: Up to and including XI 3.0 SP14, the File Adapter exclusively uses passive data connections. Starting with XI 3.0 SP15 / PI 7.0, active connections are available.

24. FTPS: Server certificate rejected by ChainVerifier

Q: After configuring a File Adapter FTPS communication channel, I see the following error message in the Adapter Monitor: iaik.security.ssl.SSLException: Server certificate rejected by ChainVerifier. What do I need to configure to get my scenario working?
A: The File Adapter checks whether a trusted certificate chain exists for the X.509 certificate of the FTP server it is connecting to. A certificate chain is considered trusted if the chain is valid and at least one of its certificates is directly trusted. If the certificate chain is untrusted, the above exception is thrown. To configure a single certificate or a certification authority's CA certificate as directly trusted, add the respective X.509 certificate to the "Trusted CAs" view of the "Key Storage" service in the SAP J2EE Visual Administrator. In most cases, this will be the CA certificate that was used to sign the server certificate.

25. FTP Proxy Support

Q: Does the File Adapter provide a means to configure an FTP proxy server used to establish any FTP connections?

A: No, this is not supported. This limitation also applies to FTPS.

26. FTP Connection Problems

Q: I have configured a File Adapter channel to connect to an FTP server, but receive one of the following error messages in the adapter monitor:

Error connecting to ftp server '<hostname>': SocketException: Connection reset

Error connecting to ftp server '<hostname>': SocketException: Connection refused

Error connecting to ftp server '<hostname>': ConnectException: Connection timed out

A: This problem is either caused by incorrect firewall / packet filter settings or an incorrect configuration of the FTP server. Also make sure that you have correctly specified the host name / IP address and port of the FTP server.

Changing the connection type from 'active' to 'passive' (or vice versa) might additionally help to work around the incorrect firewall configuration.

27. FTP Directory Listing Semantics

Q: Which semantics does the File Sender Adapter apply to determine which files to retrieve from an FTP server?

A: Up to and including XI 3.0 SP13 the File Adapter will issue an NLST command (see RFC 959) with the configured source file name scheme as first parameter to the FTP server. It will try to retrieve any of the files returned by the FTP server in response to this command.

Starting with XI 3.0 SP14 / PI 7.0, the File Adapter will issue a LIST command with the configured source file name scheme as first parameter to the FTP server. If it is able to parse the returned result (which is the case for UNIX- and Windows-style directory listings), it will retrieve any of the returned files. If it cannot parse the result, it will fall back to the pre-SP14 implementation and use the NLST command as described above.

28. sftp Support

Q: Does the File Adapter support the sftp (File Transfer over SSH) protocol?

A: No, this protocol is not supported and there are currently no plans to implement sftp support in the File Adapter.

29. File Content Conversion fieldFixedType Parameter Not Working

Q: After configuring a File Adapter channel with File Content Conversion and the fieldFixedType=byte parameter set, I noticed that this setting does not seem to have any effect.

A: The parameter's name should correctly read "fieldFixedLengthType". The documentation will be updated accordingly with XI 3.0 SP17 / PI 7.0.

30. Missing Features After Upgrade

Q: After applying a support package, I am unable to find a new feature advertised in the release notes in the channel configuration.

A: You most likely did not import the updated SAP_BASIS content for the support package.

Note 821267 - FAQ: XI 3.0 / PI 7.0/ PI 7.1 File Adapter 1


 

11. File Sender: Processing Multiple Source Directories

Q: Can the File Sender Adapter be configured to poll multiple source directories from a single channel?

A: This functionality is available starting with XI 3.0 Support Package 14 as well as PI 7.0.

12. File Sender: Quality of Service EOIO (Exactly Once In Order)

Q: How does a File Adapter sender channel handle the quality of service EOIO? Are files that match the File Name Scheme in the sender's channel configuration processed in arbitrary order?

A: The File Adapter allows you to configure the Processing Sequence of files for the Transport Protocol "File System (NFS)", which also determines the order for EOIO processing. Files can either be processed in ascending alphabetical order (Processing Sequence "By Name") or by their last modification time stamp (Processing Sequence "By Date"), where the oldest file is processed first. For the Transport Protocol "File Transfer Protocol (FTP)" files are always processed in ascending alphabetical order.

13. File Receiver: FTP Problems for Large Files with Microsoft IIS

Q: I have configured a File Adapter receiver channel to upload to a Microsoft Internet Information Server FTP server using the "Put File" mode "Use Temporary File".

When sending a large file to the receiver, I get the following error: "com.sap.aii.adapter.file.ftpFTPEx: <filename>: The process cannot access the file because it is being used by another process."

How do I solve this problem?

A: The cause of this problem is a known bug in the Microsoft IIS FTP server implementation you are using. For details, refer to:

IIS 4.0: http://support.microsoft.com/default.aspx?scid=kb;en-us;238644

There is no fix available from Microsoft. As a workaround, set the configuration parameter "Put File" to "Directly".

IIS 5.0: The problem is currently not documented by Microsoft. As a workaround, set the configuration parameter "Put File" to "Directly".

IIS 6.0: http://support.microsoft.com/default.aspx?scid=kb;en-us;828086

Apply the hotfix provided by Microsoft.

14. Memory Requirements

Q: Which memory requirements does the File Adapter have? Is there a restriction on the maximum file size it can process?

A: The maximum file size that can be processed by the File Adapter depends on a number of factors:

The most important one is the size of the Java heap, which is shared among all messages processed at a certain point in time. In order to be able to process larger messages without an out of memory error (OOM), it is recommended to increase the size of the available Java heap and/or to reduce the concurrency in the system so that fewer messages are processed in parallel.

Another factor negatively influencing the maximum message size in releases up to and including XI 3.0 SP 13 is an enabled charcter set (encoding) conversion if the message type is set to "Text".

Using the transport protocol "File Transfer Protocol (FTP)" also uses more memory for processing than the transport protocol "File System (NFS)" (up to and including XI 3.0 SP 13).

If the Message Protocol "File Content Conversion" is used in a File Sender channel, consider that not only the size of the input file affects the File Adapter's memory usage, but even more the size of the XML resulting from the conversion, which is usually a few factors larger than the original plain text file.

To reduce the memory consumption in this scenario, consider configuring the setting "Maximum Recordsets per Message" for the sender channel. This will cause the input file to be split into multiple smaller mesages.

15. Operating System Command

Q: I am having difficulties getting an external operating system command to work. What can I do to diagnose the problem?

A: Refer to note 841704.

16. File Encoding

Q: How do I correctly configure the File Encoding used by the File Adapter?

A: Generally, the message payload of an XI message is treated as UTF-8 when it needs to be converted from or into a different encoding. So, what you specify in the File Sender channel is the source encoding for an encoding conversion to UTF-8. On the other hand, in the Receiver channel, you specify the target encoding for a conversion from UTF-8. If you configure a channel's File Type setting as "Binary", no conversion will be applied.

Depending on your scenario, only some encoding settings actually make sense and lead to the expected results:

XML Files

An XML file's encoding is set in the XML header, which is later interpreted when parsing the XML, e.g. in the mapping, so there is no necessity to perform an encoding conversion in the File Adapter. As a rule of thumb, always configure the File Type parameter of a sender or receiver channel as "Binary" when reading or writing XML data.

Important: Even if you configure a File Encoding in the File Adapter channel, the File Adapter will not re-write the XML header to reflect the changed encoding, so you will probably see an XML parsing error later during the processing of the message if you specify an encoding.

Flat Files with File Content Conversion

For a File Sender channel, configure the encoding of the source file. The file will be interpreted according to the configured encoding and converted to XML with an UTF-8 encoding.

For a File Receiver channel, configure the encoding to match the encoding you would like to be written to the target flat file.

Flat Files without File Content Conversion

Whether to configure an encoding in this case depends on if you want to pass through the file "as is", e.g. within a File Sender to File Receiver scenario, or if you want to convert the file's encoding on its way through the Integration Server. For "as is" processing, configure both the sender and the receiver using the File Type setting "Binary".

To apply an encoding conversion, configure the respective source and target encoding in both the sender and receiver channel.

Important: Configuring an encoding in the receiver channel will only lead to the expected results if the payload sent to the receiver channel is in UTF-8 format (e.g., by having specified an encoding conversion in the Sender channel).

17. Empty File Encoding Parameter

Q: Which encoding is used if I configure the File Type as "Text", but leave the File Encoding parameter empty?

A: In this case, the system encoding will be used, e.g. "Cp1252" on most western Windows installations.

18. File Sender: Scheduling

Q: How do I configure a File Adapter sender channel to poll a directory at a specific time of day?

A: This is available from SP 20 in Communication channel monitoring with the feature by name 'Availability Time Planning'.

19. Secure FTP (FTPS)

Q: I would like to use Secure FTP (FTPS) with the File Adapter. Does my FTP server support this out of the box? Which configuration changes do I need to apply to my FTP server?

A: Not every FTP server supports FTPS and many that do require a configuration change to activate the FTPS protocol extension. Contact your FTP server software vendor for detailed information.

20. File Sender: Special Characters in File Names

Q: I am trying to get the File Adapter to poll a file, which contains special characters (e.g., accented characters or umlauts) in its file name. However, irrespectively of the wildcard mask I specify in the File Adapter sender channel configuration, the file does not get picked up. Which configuration setting do I need to change to get my scenario working?

A: Under certain operating system platforms, such as Solaris, the APIs used by the Java Runtime (JRE) are not Unicode-aware. Consequently, the JRE needs to be configured to correctly interpret the character set it receives from the operating system.

This is configured through the "file.encoding" system property as well as the "LANG" environment variable.

Make sure you set "file.encoding" to a character set (such as ISO-8859-1) that supports the special characters you would like to process. This system property can be configured by appending "-Dfile.encoding=<encoding>" to the Java VM parameters section of the SAP J2EE Config Tool.

Additionally, you need to set the "LANG" environment variable to a locale that supports more than 7 bits, such as "de.ISO8859-1". The encoding you specify in the LANG environment variable needs to match the encoding set via "file.encoding".

You can persistently configure the environment variable by setting it in the profile $HOME/.sapenv_$HOSTNAME.csh of the <sid>adm user: setenv LANG de.ISO8859-1. For additional details on 'How to Work with Character Encodings in PI' the following guide can also be followed: https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/com.sap.km.cm.docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79fb42

Note 821267 - FAQ: XI 3.0 / PI 7.0/ PI 7.1 File Adapter

Table of Contents


 

1. Information to Include in OSS Message

2. Read-Only File Processing

3. File Locking / Incomplete Processing

4. FTP Sender File Processing in Cluster Environment

5. File Receiver: Overwrite Protection

6. Acknowledgements

7. File Construction Mode "Add Time Stamp" and "Add Counter" in Cluster Environment

8. FTP Adapter Timeout Configuration

9. Module Development: Accessing the Source File Name from a Module

10. <RecordSetName>.fieldContentFormatting=nothing and SXMB_MONI

11. File Sender: Processing Multiple Source Directories

12. File Sender: Quality of Service EOIO (Exactly Once In Order)

13. File Receiver: FTP Problems for Large Files with Microsoft IIS

14. Memory Requirements

15. Operating System Command

16. File Encoding

17. Empty File Encoding Parameter

18. File Sender: Scheduling

19. Secure FTP (FTPS)

20. File Sender: Special Characters in File Names

21. File Sender: Performance Issues After Configuring A Large Amount Of Channels

22. File Sender: File Modification Check For Transport Protocol "FTP"

23. FTP: Active Data Connection Support

24. FTPS: Server certificate rejected by ChainVerifier

25. FTP Proxy Support

26. FTP Connection Problems

27. FTP Directory Listing Semantics

28. sftp Support

29. File Content Conversion fieldFixedType Parameter Not Working

30. Missing Features After Upgrade

31. Empty File Processing

32. File Construction Mode "Add Counter": Counter Reset

33. FTPS: "AUTH TLS" Command Unrecognized

34. FTP: Directory Specification / CWD Command

35. File Sender Error Handling

36. FTP: Adapter Polls More Frequently Than Configured

37. FTP: sun.io.MalformedInputException

38. Windows (SMB) Shares

39. FTP Sender: 550 File Not Found

40. Known Problems and Incompatibilities for FTP servers

41. Test FTP Server Connection

42. Channel prioritization using OS command

43. J2EE engine hangs with File/FTP sender channels

44. File Receiver with append mode on multiple cluster nodes

45. Receiver File Adapter stuck while using temporary file.

46. How to manually delete hanging Enqueue-Locks

47. Effect of setting Maximum Concurrency and poolWaitingTime


 

Questions and Answers

1. Information to Include in OSS Message

Q: I need to open an OSS message regarding the File Adapter with SAP. Which information should I put into the message?

A: Refer to note 854536.

2. Read-Only File Processing

Q: I have configured a File Adapter sender channel to process a source file to which it only has read-only access (e.g., because it has been written by a different OS user). The file is not processed, but the channel is nevertheless listed as working in the Adapter Monitoring. Have I encountered a bug?

A: This behavior is by design: The File Adapter sender channel does not process read-only files by default. As this is intended, you will never see an error message indicating this condition in the Adapter Monitoring or log. To enable processing of read-only files, set the "Files Locked (Read-Only)" flag for the respective sender channel.

3. File Locking / Incomplete Processing

Q: I sometimes observe that files are processed only partially, i.e., only a fragment from the file's start is converted into an XI message. Nevertheless, the file is usually archived completely. Does the File Sender Adapter honor if another process has locked a file for exclusive use?

A: Unfortunately, the J2EE 1.3 technology the File Adapter is built upon does not support file locking. This limitation affects the File Adapter's operation. Depending on whether the JRE implementation for the operating system under which the adapter runs uses mandatory file locking or advisory file locking, opening a file that is currently being written to by another process will fail or not.

If opening the file fails, no problem exists and the adapter will try to open the file each poll interval until it succeeds.

However, if opening the file is not prevented by the operating system, the adapter starts to process the file although it is still being modified. Since XI 3.0 SP11 / PI 7.0 there is a parameter named "Msecs to Wait Before Modification Check" in the advanced settings of the File Sender channel configuration to work around this issue. This setting causes the File Adapter to wait a certain time after reading, but before sending a file to the Adapter Engine. If the file has been modified (which is basically determined by comparing the size of the read data with the current file size of the input file) after the configured interval has elapsed, the adapter aborts the processing of the file and tries to process the file again after the retry interval has elapsed.

If this option is not available for the settings you would like to use, the following algorithm (to be implemented in your application) may be used to ensure that the File Adapter only processes completely written files:

Create the file using an extension, which does not get processed by the File Adapter, e.g., ".tmp"

Write the file content

Rename the file to its final name, so the File Adapter will notice its existence and pick it up

4. FTP Sender File Processing in Cluster Environment

Q: When running the File Adapter in an environment with more than a single J2EE server instance, the same file is fetched by multiple cluster nodes at once. How do I solve this problem?

A: Initially, make sure that you do NOT use the processing mode "Test" for your File Sender channel. This mode is intended to be used only for testing purposes and is not guaranteed to work in a cluster environment.

Additionally, activate the "Advanced Mode" for the respective sender channel. No further configuration changes are necessary. Alternatively, upgrade to XI 3.0 SP11 Patch Level 2 or XI 3.0 SP12. Later XI 3.0 versions / PI 7.0 are not affected.

5. File Receiver: Overwrite Protection

Q: When running a File Adapter Receiver <= XI 3.0 SP9 with a File Construction Mode "Create", target files are overwritten although the "Overwrite Existing Files" setting is not active in the Receiver channel. How do I solve this problem?

A: This problem has been corrected with XI 3.0 SP10. PI 7.0 is not affected.

6. Acknowledgements

Q: Does the File Adapter support acknowledgements?

A: You need to distinguish system acknowledgements (indicating that a message has been received by the target system) and application acknowledgements (indicating that the message has been successfully processed by the application on the receiver side).

The receiver of an XI message will only send an acknowledgement back to the sender if the sender has requested one. However, the File Adapter has no functionality that relies on the receipt of an acknowledgement, so it never requests one.

On the other hand, if a File Adapter Receiver receives a request to send an acknowledgement, it will do so for a system acknowledgement request. Application acknowledgements are not supported at all as the File Receiver has no way to determine if the written file has been correctly processed by the back-end application, which is what a positive application acknowledgement would imply.

7. File Construction Mode "Add Time Stamp" and "Add Counter" in Cluster Environment

Q: We have set up a File Adapter Receiver on an XI installation with multiple server nodes. The receiver channel is configured to use the File Construction Mode "Add Time Stamp" or "Add Counter". We have noticed, that sporadically two messages are written to the same file name. How can we work around this issue?

A: If you are using the transport protocol "File System (NFS)", apply note 869234. For the transport protocol "File Transfer Protocol (FTP)" the time stamp and counter is not guaranteed to be unique across cluster nodes, so two messages delivered to the same receiver channel at the same time on different cluster nodes might cause the target to be overwritten by the second message. If you need to create truly unique files by the receiver, use the File Construction Mode "Add Message ID".

8. FTP Adapter Timeout Configuration

Q: We are using the FTP Adapter to send or receive messages from/to an FTP server, which is sporadically experiencing connectivity problems. This causes the FTP Adapter to hang. Is there any way to configure a timeout?

A: Refer to note 849089.

9. Module Development: Accessing the Source File Name from a Module

Q: When processing a message originating from a File Adapter Sender channel, how do I access the original name of the file read by the File Sender channel from within a module?

A: See note 819761 for a HOWTO.

10. <RecordSetName>.fieldContentFormatting=nothing and SXMB_MONI

Q: We have configured a File Sender Adapter with File Content Conversion not to remove the whitespace around fixed-width fields from the input file by setting the parameter <RecordSetName>.fieldContentFormatting to "nothing". Nevertheless, the whitespace seems to be trimmed as we can see in SXMB_MONI. Have we encountered a bug?

A: SXI_MONITOR displays an interpreted version of the XML payload, which unfortunately hides the whitespace. Nevertheless, this whitespace is still present in the generated XML file as you will see if you send the message to a File Adapter Receiver that writes the payload to a file.

SAP XI/PI 7.1 ABAP Reports


 

 
 

 
 

RSALERTCATDEF

ALM: Maintaining alert categories and alert server landscape

RSBDCOS0

Execute OS command

RSQOWKEX

QOUT Scheduler: execution of registered outbound queues

RSXMB_CANCEL_NO_COMMIT_MSG

Cancel message processing for messages without commit

RSXMB_RESTART_MESSAGES

Restart of erroneous, asynchronous messages

SPROX_CHECK_IFR_ADDRESS

Check if IFR address is maintained

SRFCCONN

RFC Test

SXMS_MMF_MONITOR

display bulk message and the corresponding individual messages

SXMS_PF_LEVEL12_AGGDATA

Function module used to get aggregated performance data on RWB

SXMS_REFRESH_ADAPTER_STATUS

Refresh of outbound adapter status, only for IDoc adapter and ccBPM processes

RSEOUT00

This Report provides the maximum number of the idocs to be collected and can trigger the collected idocs at a time and places in a file.Input Parameters for this report is Idoc Numbers(from and to), Basic Type, Port Type and Maximum Number of Idocs.  Example:  If  the total number of idocs is 2600, and we provide 1500 in "maximum number of Idocs" field, then this report collects the idocs until it reach 1500.In this case total 2 idocs will generated one with 1500 and other with 1100.

XI Transaction Codes

Administration 

    AL08 -> list logged in users
    AL11 -> SAP Directories
    PFCG -> Roles
    SCC4 -> Client Administration
    SU01 -> User Maintenance
    SU10 -> User Maintenance - Mass Changes
    AL08 -> Show logged on users
    SXMB_ADM -> Integration Engine - Administration
    SWELS -> BPM trace

Alerts 

    ALRTCATDEF -> Alerts Configuration
    ALRTDISP -> Display of Alerts Created
    ALRTINBOX  ->BSP alert inbox
    ALRTPROC  -> Process Alerts

CCMS

    RZ21 -> Monitoring : Properties & Methods
    RZ20 -> CCMS Monitor Sets
    GRMG -> Generic Request and Message Generator

Configuration

    SLDAPICUST -> SLD API Customizing
    RZ10 -> edit system profile

=================

IDoc

     Configuration

          WE20 -> Partner Profiles
          WE21 -> port definition
          BD82 -> Maintain partner profile automatically
          BD54 -> Logical System Creation
          BD64 -> Maintain distribution  model
          BD71 -> Distribute customer model
          SALE -> Create  Logical System
          WE31 -> Segment Creation at Transaction
          WE30-> Create IDOC (Basic IDoc Creation)

     Adapter

          IDX1 -> idoc adapter
          IDX2 -> idoc metadata
          IDX5 -> monitor idoc adapter

     Monitoring

          WE02 -> Idoc display
          WE05 -> Idoc list
          WE07 -> Idoc statistics

          SM58 -> QRFC (SOMETIMES idocs coluld not leave the SAP sending system)
          SM50 -> Processes (are there enough in case of masss load)
          SMQ2-> Inbound Queues 

     ALE Administration

          SALE -> Define ALE logical system
          WE20 -> Partner Profile
          WE21 -> IDoc port
          WEDI -> Mainatain menu for EDI related activities
          BALE -> Main menu for ALE related activities

     Idoc Definition

          WE31 -> Segment editor
          WE30 -> Idoc editor to create and extend Idoc type
          BD53 -> Reduce Idoc types for master data
          WE60 -> Idoc documentation (structure and segment definition)
          WE61 -> Idoc documentation (control, data and status records)

     Testing

          WE19 -> Test tool for IDOC
          WE12 -> Create an inbound into an outbound
          WE16 -> Process an incoming Idoc file
          WE17 -> Process an incoming status file

     Reprocessing IDocs

          BD87 -> Outbound
          BD88 -> Inbound

     Packaging

          IDXPW -> to activate the IDOC message package
          IDXP -> to monitor the message packages

     Search

          WE09-> Search for IDocs by Content

=================

General

    SXMB_IFR -> Start Integration Builder
    SXI_CACHE ->  XI Runtime (from IB) cache

    SXI_SUPPORT -> Test all the Repository and Directory Objects

Monitoring

    SMICM -> ICM Monitor
    SXMB_MONI -> Integration Engine - Monitoring
    SXI_MONITOR -> XI: Message Monitoring
    SXMB_MONI_BPE -> Process Engine - Monitoring
    SMGW -> Gateway Monitor
    ST06 -> Operating System Monitor. cpu memory filesystem machine status
    SM37 -> Job Monitoring
    ST03 -> workload. 
    SM21-> System logs 

Message Packaging

    SXMS_BCM -> define Configuration types
    SXMS_BCONF -> Global Package Configuration
    XMSPKSTATCNF -> Package Statistics Configuration
    XMSPKSTATMON -> Package Statistics Monitoring

Queues

    SMQ1 -> qRFC monitor - outbound message queue
    SMQ2 -> qRFC monitor - inbound message queue
    SMQ3 -> qRFC monitor (save queue entry)
    SMQS -> to register the destination in QOUT scheduler
    SMQR -> to register the queue

RFC

    SM58-> Asynchronous RFC Error Log
    SM59-> RFC Destinations

Troubleshooting

    SLDCHECK -> Test SLD Connection
    ST22 -> ABAP dump analysis
    SM21 -> Online System Log Analysis
    SM50 -> Work process overview
    ST11 -> Error Log Files

-----

uncategorised

SE09/SE10 -> Change Request(Transport Organiser)
SE11 -> Data dictionary
se37 -> mantain funcion modules
SICF -> http server configuration
table TSTC -> sap transactions codes
table TSTCT -> transaction descriptions
STMS -> transports management
SPAM -> apply ABAP support packages
SPAU -> manage objects after apply support packages
SE01 -> manage change requests

SM12 -> Table locks
SE38 -> ABAP Editor
SE11 -> ABAP Dictionary
SPROXY-> ABAP Proxy Generation
SE80 -> Object Navigator
ABAPDOCU -> ABAP Documentation and Examples
SE24-> Class Builder

RZ70-> SLD Data Supplier

SE16-> Data Browser
SE93-> Maintain Transaction Codes
SM30-> Call View Maintenance
SM02-> System Messages

SWF_XI_PBUILDER -> for Detail BPM Process
SWF_XI_CUSTOMIZING -> transaction to check prerequisites for integration processes.
WEOUTQUEUE -> to start the queue processing
SWF_XI_ADM_BPE-> Start or Stop BPE
SWF_XI_ADM_BPE_DISP-> Display BPE Status

SXI_MAPPING_Test--> Test ABAP Mapping 

LIST OF SAP XI TABLES

ABAP

ABAP schema Database Transaction Codes

  
SE11

   SE16

  

  

SXMSPMAST

XI Messages - Master

SXMSPHIST

XI Messages - Master History table

SXMSPEMAS  

Integration Engine: Enhanced Message Queue (Master)

SXMSPVERS 

Integration Engine: Message Version

SXMSPERROR

XML Message Broker: Message Queue (Incorrect Entries)

SXMSCLUP 

Property Cluster

SXMSCLUR

Resources Cluster


*N.B.,
   The Switch equivalents for the 7 tables above have the suffix 2, e.g.
SXMSPMAST2 


Mapping Related Tables

  

SMPPMAP3

Mapping Runtime: Mapping

SMPPREL3

Mapping Runtime: Mapping Relation

SMPPSPLIT

XI Mapping: Merge and Split


Performance

SXMSPFRAWH

Integration Engine - Header Table for Raw Data for Performance Evaluation

SXMSPFRAWD 

Integration Engine: Data Table for Raw Data for Performance  


 

ccBPM / BPE tables

  

  

SWFRXIHDR

XI Adapter: Header Data for a Message Object

SWFRXICNT 

XI Adapter: XML Container for a Message Object

SWFRXIPRC

XI-Adapter: Process-Dependent Message Data

IDoc

IDXNOALE

suppress IDoc Adapter acknowledgements


Java 

Adapter Engine

  

  

XI_AF_MSG

AF XI Message

XI_AF_MSG_AUDIT

audit log entries

BC_MSG

AF PI 7.1  Message

BC_MSG_AUDIT

PI 7.1 audit log entries (**)

**N.B.,

Integration Builder

  

  

SAP_<SID>DB.XI_REP*   

Repository objects

SAP_<SID>DB.XI_DIR*    

Directory objects

Integration Builder versioning

  

  

SAP<SID>DB.XI_P*

Repository versioning information

SAP<SID>DB.XI_Q*

Directory versioning information



 

IDoc Statuses

No.

Description

01

IDoc created

02

Error passing data to port

03

Data passed to port OK

04

Error within control information of EDI subsystem

05

Error during translation

06

Translation OK

07

Error during syntax check

08

Syntax check OK

09

Error during interchange handling

10

Interchange handling OK

11

Error during dispatch

12

Dispatch OK

13

Retransmission OK

14

Interchange Acknowledgement positive

15

Interchange Acknowledgement negative

16

Functional Acknowledgement positive

17

Functional Acknowledgement negative

18

Triggering EDI subsystem OK

19

Data transfer for test OK

20

Error triggering EDI subsystem

21

Error passing data for test

22

Dispatch OK, acknowledgement still due

23

Error during retransmission

24

Control information of EDI subsystem OK

25

Processing despite syntax error (outbound)

26

Error during syntax check of IDoc (outbound)

27

Error in dispatch level (ALE service)

29

Error in ALE service

30

IDoc ready for dispatch (ALE service)

31

Error - no further processing

32

IDoc was edited

33

Original of an IDoc which was edited

34

Error in control record of IDoc

35

IDoc reloaded from archive

36

Electronic signature not performed (timeout)

37

IDoc added incorrectly

38

IDoc archived

39

IDoc is in the receiving system (ALE service)

40

Application document not created in receiving system

41

Application document created in receiving system

42

IDoc was created by test transaction

50

IDoc added

51

Error: Application document not posted

52

Application document not fully posted

53

Application document posted

54

Error during formal application check

55

Formal application check OK

56

IDoc with errors added

57

Test IDoc: Error during application check

58

IDoc-Copy from an R/2 connection

60

Error during syntax check of IDoc (inbound)

61

Processing despite syntax error (inbound)

62

IDoc passed to application

63

Error passing IDoc to application

64

IDoc ready to be transferred to application

65

Error in ALE service

66

IDoc is waiting for predecessor IDoc (serialization)

67

Not used

68

Error - no further processing

69

IDoc was edited

70

Original of an IDoc which was edited

71

IDoc reloaded from archive

73

IDoc archived

74

IDoc was created by test transaction

SAP Developer Network SAP Weblogs: SAP Process Integration (PI)