Tractor is Pixar’s
Tractor distributes tasks to a farm of execution servers. It manages large queues of concurrent jobs from many users, enforcing dependencies and scheduling policies.
Tractor is designed to be a compact system, relative to the complex role that it plays:
- Tractor-Engine: The central job queue manager, self-contained single install. Low overhead operation with engineering emphasis on high throughput job distribution for large farms.
- Tractor-Blade: Simplified “plug-and-play” server deployment on any size farm.
- Tractor-Dashboard Web UI for ubiquitous access to job feedback and farm status.
- Scripting and command-line tools for wranglers and advanced users.
- Component architecture based on common web technologies.
Tractor can drive all of the computational tools used in modern VFX and animation pipelines. It is used to run everything from rendering and compositing to simulation, transcoding, archiving, database updates, code builds, online asset delivery and notifications. Tractor can launch almost any executable available on your Linux, Windows, and Mac OS X systems.
A tractor Job describes what needs to get done. Jobs are comprised of Tasks, which represent the steps required to complete the job. A job definition describes the resources that are required for each task, and whether the task will run serially or in parallel with other tasks in the job. Jobs can be created by hand, by scripts, by the supplied Python API, or by plug-in job generators in content-creation applications.
Jobs are spooled to the Engine, the central process that maintains the job queue. Tractor-engine selects an appropriate Tractor Blade execution node on the farm for each command, using an abstract keyword matching scheme and resource limit restrictions. The engine also takes care of enforcing dependencies between tasks, as well as finding opportunities for parallel execution, allowing different jobs or independent parts of a single job to run on different servers concurrently. Tractor provides flexible schemes for job prioritization, compute resource access controls, dynamic server availability, and user limit policies.
The Tractor Dashboard web interface provides users and administrators with simple feedback on job status, errors, and overall system status. It also provides drill-down access to detailed information on job structure, attributes, and output logs, as well as detailed data on the blades. The Tractor Query tools provide access to more complex relational queries into the job queue and past execution data. There are query tool interfaces in the web Dashboard, or through a Python API, or on the command line to provide wranglers with quick diagnostics or to build custom reports.
Tractor components around the network communicate using common web-based technologies such as HTTP, JSON, and AJAX. Studios can develop their own custom extensions or external applications that communicate with Tractor using these same standard tools.
Tractor 2 Highlights
- Powerful scripting interfaces
- Flexible resource-sharing controls
- Python module extensions
- Minimal computational footprint
- Streamlined installation
- Major user interface enhancements
- Open web standards
- HTML 5 support
Tractor is Pixar’s distributed processing solution for rendering and other network intensive applications. Derived from years of experience in managing the demands of the worlds largest render farms, Tractor is specifically engineered to address the extreme scalability and performance requirements of today’s multi-core networks.
Tractor’s queuing engine is capable of dispatching over 500 tasks per second. Tractor utilizes proven, open web standards and is highly customizable. Systems administrators can easily add Python extension modules to Tractor-Blades and customize the specifics of how various tasks are executed.
Functionally, Tractor consists of three components:
- The Tractor-Engine maintains the central job queue and dispatches tasks.
- The Tractor-Blade is a Python based execution server running on each remote node.
- The Tractor-Dashboard is a customizable web browser user interface providing centralized control of the Tractor-Engine.
Full Integration with RenderMan
RenderMan Studio includes out-of-box support for Tractor to facilitate job submissions directly from a session of Maya. Submit jobs locally or remotely, and monitor them with Tractor’s Dashboard. For more information check out the Tractor workflow with RMS.