This chapter looks at advanced options for configuring and tuning thread pools, described hazards to watch for when using the task exeuction framework, and offers some more advanced examples of using
Executor framework offers substantial flexibility in specifying and modifying execution policies, not all tasks are compatible with all execution policies. Types of tasks that require specific execution policies include:
- Dependent tasks
- Tasks that exploit thread confinement
- Responsive-time-sensitive tasks
- Tasks that use
Thread pools work best when tasks are
independent. Mixing long-running and short-running tasks risks “clogging” the pool unless it is very large; submitting tasks that depend on other tasks risks deadlock unless the pool is unbounded.
Some tasks have characteristics that require or preclude a specific execution policy. Tasks that depend on other tasks require that the thread pool be large enough that tasks are never queued or rejected; tasks that exploit thread confinement require sequential execution. Document these requirements so that future maintainers do no undermine safety or liveness by substituting an imcompatible execution policy.
If all threads are executing tasks that are blocked waiting for other tasks still on the work queue, it is called thread starvation deadlock, and can occur whenever a pool task initiates an unbounded blocking wait for some resource or condition that can succeed only through the action of another pool task.
ThreadDeadLock illustrates thread starvation deadlock.
Whenever you submit to an
Executor tasks that are not independent, beware of the possibility of thread starvation deadlock, and document any pool sizing or configuration constraints in the code or configuration file where the
Executor is configured.
Thread pools can have responsiveness problems if tasks can block for extended periods of time, even if deadlock is not a possibility. A thread pool can become clogged with long-running tasks, increasing the service time even for short tasks.
One technique that can mitigate the ill effects of long-running tasks is for tasks to use timed resource waits instead of unbounded waits. Most blocking methods in the platform libraries come in both untimed and timed versions, such as
The ideal size for a thread pool depends on the types of tasks that will be submitted and the chracteristics of the deployment system. Thread pool sizes should rarely be hard-coded; instead pool sizes should be provided by a configuration mechanism or computed dynamically by consulting
For compute-intensive tasks, an Ncpu - processor system usually achieves optional utilization with a thread pool of Ncpu + 1 threads. For tasks that also include I/O or other blocking operations, you want a larger pool, since not all of the threads will schedulable at all times.
Given these definitions:
The optimal pool size for keeping the processors at the desired utilization is:
ThreadPoolExecutor provides the base implementation for the executors returned by the
newScheduledThreadExecutor factories in
Executors. If the default execution policy does not meet your needs, you can instantiate a
ThreadPoolExecutor through its construtor and customize it as you see fit.
The core pool size, maximum pool size, and keep-alive time govern thread creation and teardown.
- The core size is the target size; the implementation attempts to maintain the pool at this size even when there are no tasks to execute; and will not create more thread than this unless the work queue is full
- The maximum pool size is the upper bound on how many pool threads can be active at once
- A thread that has been idle for longer than keep-alive time becomes a candidate for reaping and can be terminated if the current pool size exceeds the core size
In chapter 6 we saw how unbounded thread creation could lead to instability, and addressed this problem by using a fixed-sized thread pool instead of creating new thread for every request. However, this is only a partial solution; it is still possible for the application to run out of resources under heavy load, just harder.
If the arrival rate of new requests exceeds the rate at which they can be handled, requests will still queue up. With a thread pool, they wait in a queue of
Runnables managed by the
Executor instead of queueing up as threads contending for the CPU. Representing a waiting task with a
Runnable and a list not is certainly a lot cheaper than with a thread, but the risk of resource exhaustion still remains if clients can throw requests at the server faster than it can handle them.
ThreadPoolExecutor allows you to supply a
BlockingQueue to hold tasks awaiting execution. There are three basic approaches to task queueing: unbounded queue, bounded queue, and synchronous handoff.
newCachedThreadPool factory is a good default choice for an
Executor, providing better queueing performance than a fixed thread pool. A fixed size thread pool is a good choice when you need to limit the number of concurrent tasks for resource-management purposes, as in a server application that accepts requests from network clients and would otherwise be vulnerable to overload.
When a bounded work queue fills up, the saturation policy comes into play. The sturation policy for a
ThreadPoolExecutor can be modified by calling
Several implementations of
RejectedExecutionHandler are provided, each implementing a different saturation policy:
AbortPolicy: default, causes
executeto throw the unchecked
RejectedExecutionException, the caller can catch this exception and implement its own overflow handling as it sees fit
CallerRunsPolicy: implements a form of throttling that neither discards tasks nor throws an exception, but instead tries to slow down the flow of new tasks by pusing some of the work back to caller
DiscardPolicy: silently discards the newly submitted task if it cannot be queued for execution
DiscardOldestPolicy: discards the task that would otherwise be executed next and tries to resubmit the new task
Choosing a saturation policy or making other changes to the execution policy can be done when the
Executor is created.
We could also use
Semaphore to bound the task injection rate.
Whenever a thread pool needs to create a thread, it does so through a thread factory. The default thread factory creates a new nondaemon thread with no special configuration. Specifying a thread factory allows you to customize the configuration of pool threads.
MyThreadFactory will instantiate a new
MyAppThread with a pool-specific name, so threads from each pool can be distinguished in thread dumps and error logs.
Most of the options passed to the
ThreadPoolExecutor constructors can also be modified after construction via setters.
Executors includes a factory method
unconfigurableExecutorService, which takes an existing
ExecutorService and wraps it with one exposing only the methods of
ExecutorService so it can not be futher configured.
ThreadPoolExecutor was designed for extension, providing several “hooks” for subclasses to override -
terminated - that can be used to extend the behavior or
TimingThreadPool shows a custom thread pool that uses
terminated to add logging and statistics gathering.
If we have a loop whose iterations are independent and we don’t need to wait for all of them to complete before processing, we can use an
Executor to transform a sequential loop into a parallel one:
Sequential loop iterations are suitable for parallelization when each iteration is independent of the others and the work done in each iteration of the loop body is significant enough to offset the cost of managing a new task.
Loop parallelization can also be applied to some recursive designs. The easier case is when each iteration does not require the results of the recursive iteration it invokes.
We define a “puzzle” as a combination of an initial position, a goal position, and a set of rules that determine valid moves.
From this interface, we can write a simple sequential solver that searches the puzzle space until a solution is found or the puzzle space is exhausted.
Node represents a position that has been reached through some series of moves, holding a reference to the move that created the position and the previous
SequentialPuzzleSolver shows a sequential solver for the puzzle framework that performs a DFS of the puzzle space.
ConcurrentPuzzleSolver does not deal well with the case where there is no solution. One possible solution is to keep a count of active solver tasks and set the solution to null when the count drops to zero:
Executor framework is a powerful and flexible framework for concurrently executing tasks. It offers a number of tunning options, such as policies for creating and tearing down threads, handling queued tasks, and what to do with excess tasks, and provides several hooks for extending its behavior. As in most powerful frameworks, however, there are some combinations of settings that do not work well together; some types of tasks require specific execution policies, and some combinations of tunning parameters may produce strange results.
(To Be Continued)