Solution 1. Definition of the term: How is the term "archiving" used in the context of the GDPdU?
The term "archiving" is very frequently used with different meanings in discussions regarding the GDPdU. This frequently results in uncertainty and misunderstandings, which the following definition of the term is designed to prevent. The term "archiving" can be used to mean the following:
If the term "archiving requirement" is used, it actually refers to a "safekeeping requirement". It is not necessary to archive data or documents actively (in the sense of the alternative meaning of the term) if it is possible to leave them in the generating system for the entire period during which they are to be kept in safekeeping. However, data is deliberately archived for safekeeping in most cases. An example of this is contained in Section III of the communication from the German Federal Ministry of Finance dated July 16, 2001. It states that original digital documents are to be archived on data carriers that can be evaluated automatically.
In this case, the data is extracted from the production system and stored on an external archive system. An example of this is contained in section I.3 of the Ministry's communication. It states that, in the case of data archived before January 1, 2002, they (that is: The tax authorities) cannot demand that this data [...] be transferred to this data processing system again [...]. In this case, it is assumed that there are no options to access the data in the archive system from the production system.
- SAP data archiving
When you use SAP data archiving, the data does not leave the production system. It is only transferred from the database to the file system and therefore has the same place value as the residual data in the database. Data archiving means changing from relational to object-oriented data storage, which results in some differences when you access the data. For example, archived single documents can generally be displayed faster than documents from the database. On the other hand, there are disadvantages when you evaluate (report) archived data. However, data archiving is not designed for this type of data access. A specially developed tool for the drilldown reporting of historical data is provided for this purpose with SAP BW.
Therefore, SAP data archiving is not a special case of archiving in the sense of the GDPdU because the data does not leave the production system. However, with SAP data archiving, you can indeed fully comply with the requirement to store data for safekeeping.
- Storage in an archive system
In the SAP environment, you can store or optically archive both DART extracts (see below) and archive files from the file system of the production data processing system in an archive system using an interface. Here, the focus is the technical aspect of the secure storage of data over a longer period, which does not play a major role in the GDPdU discussion, since this step does not change the access options.
2. Why is data archiving necessary?
OLTP (Online Transaction Processing) systems, including SAP R/3 systems, are used for fast processing and optimization of the different business processes within a company. Fast processing of daily business data requires streamlined systems, while optimization requires an exact knowledge of the business processes as well as the ability to interrupt these processes using the relevant functions. Most of these functions are only available for open business processes because closed business processes generally cannot be changed.
To provide exactly those functions required for the data, while keeping the systems streamlined and the response times short, we offered our customers the option of separate data retention for open and closed business processes in one system a long time ago. Data archiving is used to extract the data for closed business processes from the database and write this data to the file system. By helping to restrict the data volume in a system to a reasonable amount, data archiving contributes greatly to reducing the operating costs of a system. A possible loss of accessibility as a result of archiving is accepted, since closed business processes are no longer the focus of daily work.
If the costs involved in accessing and displaying archived data were the same as they are for current data on the database, the cost and performance advantages that result from data archiving would be partially or even completely negated.
3. Is it possible to access archived data?
Data archived in a certain system always remains assigned to that system after archiving and can only be evaluated from this system. You can no longer change archived data; but any other kind of access is possible in principle.
Access to archived data must be separately programmed. No provision is generally made for evaluation (reporting) because a more suitable tool is provided with SAP BW for this type of data access. Programs that allow access to archived data generally require an archive index to ensure efficient data retrieval. An archive index places exactly the same load on the database as every other index. The ratio of index data to actual interchange data is typically 40% to 60%, provided that the data is still in the database.
In addition to streamlining the actual datasets, the main result of the reduced database load due to data archiving is a reduced number of access paths (index entries) for the data. It therefore makes sense not to use access paths for closed business processes after they have been archived. We always aim to discuss the need for archive access with customers and implement access options if required. Archive access must be implemented technically by SAP and supported by customers with the creation of appropriate indexes. However, the benefits of archiving generally decrease with each access option implemented.
4. What is the role of archived data in the context of the GDPdU?
With direct and indirect access to an SAP system, the archived data is also available to the auditor. However, this does not involve a separate archive system. The access functions comprise all of the required evaluation programs for the data of closed business processes that are deemed necessary in agreement with our customers.
5. How does an archive file differ from a DART extract?
In contrast to an archive file created with SAP data archiving, a DART extract represents a "data reserve" that can also be evaluated outside of the original system.
Therefore, a DART extract, together with its possible evaluations, corresponds more to the concept of an "archive system" that frequently occurs in the context of the GDPdU and is intended to fulfill the requirement of data evaluation after a system change, for example (Ministry communication of July 16, 2001, section I, 3b). In contrast, this is not possible with data archiving.
DART exrtact datasets that are subsequently created from archived data may be incomplete. This is associated with the fact that the reload program only processes entries from a predetermined record of tables. These tables are listed in the program documentation of the reload program (transaction FTWB).
With the selection of the data contained in the DART extract (data catalog) as defined in close cooperation with the GDPdU working group of the DSAG (German SAP User Group), customers are supported in their obligation to distinguish between tax-relevant data and other data (Question I.5 of the Ministry's Questions and Answers catalog). There is no distinction between tax-relevant data and other data with data archiving, since all data is archived in most cases. This selection does not represent any restriction of optional access (Question I.6 of the Questions and Answers catalog). The range of data contained in the data catalog is subject to updates based on practical insights from government tax audits.
If the tax auditor requests data in data carriers, this data is taken from one or more extract views, depending on the auditor's requirements. The load on the original system is reduced, as the data is accessed in extracts rather than in the original system.
6. What are the implications for SAP data archiving?
Since SAP data archiving allows you to delete data for closed business processes from the database and thus monitor the growth of the data volume, you can use high-performance online evaluation functions for current data, for example those provided with the Audit Information System (AIS) or comparable tools, while the original system load remains moderate. The requirement for providing the same access options for current and archived data could only be fulfilled by reducing the access functions for current data, without endangering system availability. However, as only data from closed business processes is archived in most cases, such a requirement is not useful from our perspective.
An enhancement of the access functions for (archived and current) data in accordance with the GDPdU would only be possible if minimum requirements were to be defined, which would then apply to all systems, regardless of the existing scope of functions. Enhancement is not possible in the case of lower releases.
In addition, you cannot reload archived data into the production system to enhance evaluation options to meet the GDPdU requirements because the data has not left the system. Also, the data would exist in the system twice after it was reloaded into the database and this would result in evaluation errors. As well as the risk of the incorrect evaluations, technical problems may occur with indexes that are flagged as "unique", which prevent reloading. Furthermore, archived data can only be interpreted correctly from the archive after a release upgrade because the valid field information is only stored here.
In the case of an SAP system with data in both the database and archive, you have one system with restricted evaluation options (with reference to the archive). The tax authority can only demand an evaluation based on the existing evaluation options in the system (Question I.3 of the Questions and Answers catalog).
We share the opinion, expressed in various publications, that an archive system separated from a production system as a reserve for tax-relevant data with certain minimum requirements standards is a good solution in this situation (Question III.11 of the Questions and Answers catalog). We provide this type of archive system in the form of DART, which meets the requirements of the GDPdU with regard to data carrier lending. Our enhancement of the access functions (for SAP data archiving) are tailored to suit the needs of our customers.