High Throughput Computing (HTC)

You can request access by filling out this form.

The Illinois Campus Cluster program is delighted to introduce HTC, a high-throughput computing service powered by the open-source workload manager, HTCondor. This service, partially funded by the Illinois Computes program, offers researchers free access to a pool of computational resources. These resources primarily consist of retired campus cluster nodes and donated hardware from projects like Compute for Humanity. HTC is currently in open-beta.

Understanding HTCondor

HTCondor is an open-source workload management system designed for high-throughput computing (HTC) environments. It specializes in managing and scheduling many compute-intensive jobs across diverse and distributed resources.

Technical Capabilities of HTCondor

• Resource Pooling: HTCondor has a much looser structure for its execute nodes (compute nodes) allowing for contributing servers to be less tightly coupled than a traditional HPC cluster
• Job Queuing and Scheduling: Users submit jobs with specific requirements (CPU, memory, etc.). HTCondor matches jobs to available resources based on priorities and policies.

Benefits of Using HTC

• It is free!
• Increased efficiency by utilizing idle computing resources.
• Cost-effectiveness by leveraging shared resources.

Is HTC Right for You?

• SLURM: Primarily designed for High Performance Computing (HPC). It’s optimized for tightly-coupled parallel jobs with high resource requirements (cores, memory, I/O) running on dedicated clusters, like the Campus Cluster.
• HTCondor: Focuses on High Throughput Computing (HTC). It manages a wider variety of workloads, including independent tasks and loosely coupled jobs. It works well with heterogeneous servers and can scavenge resources from underutilized servers.

The HTC service is free to use.