Legacy ID:KA411658
In CO there are two kinds of ETLs:
- Service ETLs
- Batch ETLs
Service ETLs are modules that continuously extract and load data into CO
Basically there are these parameters that tell you the frequency of extraction and loading data.
Not all these parameters can be changed in ETL configuration.
- Extraction period: this is the frequency of the data extraction. By default, this is every 15 minutes, starting from minute 0 of each hour. So the extraction is, for example, at 2:00, 2:15, 2:30 and so on. So, if you start an ETL for example at 2:08am, the first extraction will be at 2:15am.
- Saver period: this is the frequency in which data will be loaded into CO
- Hierarchy Import Period: this defines how often you want to import hierarchy information that will be visible in Workspace tree
Batch ETLs are modules that extract and load data (performance, configuration and relationship if supported) in every run.
The scheduling of these ETLs can be configured as needed (daily, weekly, hourly, ...)
There is another differences between these two kinds of ETL:
- Batch ETLs run as separated process: when a batch ETL start, the scheduler run it as a separated process. The ETL modules written in Java use a separated Java Virtual Machine. If you need to increase the heap size for these kind of ETLs (batch written in java) you can change the property ETL_HEAP_SIZE in customenvpre.sh
- Service ETLs run into the same scheduler process. If you need to increase the heap for these ETLs, you can change the property SCHEDULER_HEAP_SIZE in customenvpre.sh
Please reference on official documentation in
Sizing and scalability considerations for the ETL Engine server paragraph in order to have more details about it.