I just returned from the 2013 Teradata user group Partners Conference in Dallas. One of the technical topics that I presented at the conference was throttle rules and how they work. Throttles provide concurrency control in TASM, both on the EDW and the Appliance platforms. I'd like to share few points about enhancements to throttles in Teradata 14.0 and 14.10 that I discussed at the conference, in particular automatic throttles.
By the way, I'd like to thank all of you that were at the Partners Conference and who came up and introduced yourself and passed on positive feedback about my blog postings in Developer Exchange. It was very encouraging for me to hear from so many of you who are reading these comments and finding value in them.
Concurrency control, whether for queries or utility jobs, is enforced in Teradata by means of throttles. Throttles are rules within the database, and for the most part are available to all the Teradata family platforms. For example, both the 1xxx and 2xxx platforms as well as the 5xxx and 5xxx systems benefit from having system throttles and utilility throttles available on both SLES 10 and SLES 11 operating systems. In addition, once you get to SLES 11, Appliance platforms will have both workload throttles and group throttles available, which previously have only been available on the EDW.
There are several new capabilities in the concurrency control area when you get to Teradata 14.0 or 14.10, and this includes several throttles rules that are automatically created for you. This posting explores these automatic throttle rules.
Automatic Throttles on the Appliance Platform
Starring in 14.0, if you are on the 1xxx or 2xxx series hardware (the Appliance platform) on either SLES 10 or SLES 11, you will have two automatic throttles that you can see in your Workload Designer screens:
- GeneralQuery throttle places a limit of 52 on concurrent requests systemwide.
- OneSecondQuery throttle has a limit of 30 for all Low, Medium or High queries that have an estimated time greater than 1 second.
The purpose of these throttles is to give you a sensible starting point for managing concurrency on your platform, even if you are unfamiliar with workload management techniques. These default throttle rules are created during the migration process that takes you to the 14.0, or the 14.10 release. These are default rules, but can be modified or even deleted if you wish.
These default rules are not included on the EDW platforms.
One approach to modifying these two default rules on the Appliance is to limit the impact of the GeneralQuery throttle, so single and few AMP queries can always run without being delayed. Follow these steps to accomplish that modification:
- Clone FirstConfig
- Edit Cloned Ruleset
- Click on Throttles Category
- Click on GeneralQuery
- Click on Classification Tab
- Choose Query Characteristics
- Click on Add
- Choose AMP Limits
- Include only queries that use all AMPs
- OK
- Save
Automatic Throttle to Manage SQL-H queries
A new feature in 14.10 is an automatic throttle that focuses on requests that are using SQL functions used for Hadoop communication. These are referred to as SQL-H queries. If a throttle rule already exists in the rule set and it limits SQL-H queries to no more than 20 at a time, no action will be taken. However, if no such throttle exists, then an automatic throttle with a limit of 2 is added to the rule set. This applies to both EDW and Appliance platforms.
The purpose of this automatic throttle is to prevent over-use of memory that is possible if a large number of these functions run at the same time.
The throttle is created at the time the rule set is activated, but only in the cases where there is not already a rule in place to manage these types of functions. The ability to classify to workloads based on function name is a new feature in 14.10, so the migration to routine looks for throttles based on Hadoop communication functions, and only if it does not find it does it create the automatic throttle.
This automatic throttle is not recorded in the TDWM database and will not be visible in Viewpoint. It is only present in the TDWM rules cache. But it can be seen in tdwmdmp or using the PMPC APIs.
It’s rule name is Default_load_from_hcatalog, and it carries a rule ID of 800000. It cannot be deleted.
Conclusion
Automatic throttles are part of the evolution of workload management in Teradata. In current software, there are several attempts to make set up simpler for you and protect you from things you may not have thought of . Automatic throttles are one example of that direction.