Deadline For Maya

From Emily Carr University Wiki
Jump to: navigation, search

return to Animation Resources

Contents

What is Deadline?

Deadline is a render farm management program. It allows a single scene file to be efficiently rendered over multiple computers via the local area network. For this reason, Deadline requires all project files be organized and accessible on our network storage space, the R:\ render drive. For our farm we have a number of dedicated render machines and the lab computers. The lab computers are set up to render only when they are idle, so when someone starts using the computer, the frame that the computer is currently rendering will cancel so that person may properly use the computer. When a computer is not active for 15 minutes, Deadline will detect that computer and it will start rendering again.

Render Farm Preparation

Before you go about submitting your project to the render farm, consider these points:

  • Remove spaces and special characters (<>:"/\|?*) from folders and file names. Change "My project" to "Myproject" or "My_project" and "my file.ma" to "myfile.ma" or "my_file.ma".
  • Remove any extra geometry. Only objects that will be directly in front of the camera need to be present. Do not hide objects, delete them from the scene. This may involve creating a new scene for each of your shots.
  • Clean and simplify your scene to the best of your ability. In Maya, consider running File>Optimize Scene Size with "remove duplicate shading networks" and "remove unknown nodes" checked on. Go to the Hypershade window and go to Edit>Delete Unused Nodes to remove unused materials.
  • Cache simulations. If you have simulations (fluid, cloth, hair), you need to bake any simulation to a cache file. This will solve jittery hair and cloth. Go here for help on how to create an ncache: | Create new ncache. Note that if you make a change to animation that the simulation relies on, the cache will have to be recalculated. Hint: use mcx and 1 file per frame.
  • Save revisions of your scene. Save a “render” version that’s been cached, imported references, and optimized, and a “live” version for continued editing.
  • Ensure ALL project media, referenced files, textures, caches, and other files needed for rendering are placed within your project structure.
  • Optimize your scenes regularly. If you made changes since the last optimization, then you should optimize again.
  • Avoid using unnecessarily high-poly models. Large amounts of polygons can add a lot to render times.
  • Use only as many lights as absolutely necessary. Lights significantly add to render times.
  • Keep file names under 256 characters.
  • Check all render settings. Ensure your render settings are at production quality, but not so high as to cause inflated render times. For Arnold renders, this means not setting the Camera (AA) sampling rate too high. 3 or 4 is good for production quality renders. The other sample rates (Diffuse, Specular etc...) should be adjusted as needed according to the needs of your scene. Check out the Arnold documentation for useful information regarding what each sample rate is for here.
  • Do not render in jpeg's. Jpeg uses lossy compression, meaning it does not retain all the data in your image. It also degrades every time you save it. Use png's or exr's (or any other format that is lossless).
  • Don’t set reflections to be too deep. Don’t make mirrors that reflect mirrors that refract through glass that reflect off mirrors, etc. Tune the reflection depth to what each scene needs.
  • Consider converting any .psd textures to an efficient flattened file format like png, jpg, tif, tga etc. Use large 4k textures only when necessary, otherwise stick with smaller 2k, 1k, or 512x512 textures.
  • Consider breaking your scene up into layers, isolating moving and non-moving elements.
  • Use batch render to test render times. Note that initiating a Maya viewport render will not always accurately gauge your render times if you have any render layers or AOVs / Render Passes active. The Maya viewport render only renders one layer at a time and one AOV at a time. If you really want to see how long a frame is going to take you have to try a batch render of a few frames.
  • Import references. Optional but recommended. Importing your referenced models and then optimizing your scene is easier than optimizing your references one at a time. Be sure to save a new version once you import your references! Use the file with imported references to render and continue editing in the primary file.

Submitting a Job

Step One - Login to your personal account or PCoIP
  • Make sure that you are logged out of the Student user account (Start Menu>User Icon>Log Out).
  • Click on "Other User" and login to your personal account, using your myEC credentials (first initial lastname, myEC password).
  • The Student account no longer has access to the Render Drive. In order to use the Render Farm, you must use your personal account.
  • You can submit to the Render Farm over PCoIP. See How to access the Remote PC's (PCoIP) for instructions.


Step Two - Copy All Project Materials To Render Drive
Render Drive Location
Proper Folder Structure
  • Maya projects should use the default project folder organization created via File/Project Window.
  • Navigate to the Render Drive: Open Windows Explorer and click on the computer name with your username appended to it, or click the drop down menu. Click on Render (R:).
    • The Render Drive has changed format; Each user now gets their own directory, so you will only see your own files. This eliminates the clutter of the dumping ground that used to be the Render Drive.
  • Move your entire project folder (minus any unnecessary renders that might be in the ‘images’ folder) to your folder on R:\.
    • If you are using PCoIP to submit your render, then use OneDrive to transfer your files to the remote computer from your personal device. You get 1 terabyte of cloud space with OneDrive through ECU (login is also with your myEC credentials).
  • Refer to the images on the right of the page for clarification.


Step Three - Open Project From Render Drive to Verify References, Paths & Render Settings
Example Render Settings
  • Once your files have successfully copied to R:\, use File>Set Project to set the project directory to the project folder on R:\., then open your scene.
  • Verify that texture and other references are pointing to the proper, relative locations on R:\ & that all referenced files have indeed been copied to the render drive. Any lingering references to files on C:\ or S:\ will cause your render to error. Go to File>Reference Editor to point any referenced files you may have to the correct location on R:\. Another option would be to import all of your references into your scene. This will eliminate possible reference errors, though be sure to optimize your scene when you do this, and save it off as a different version.
  • Missing textures is the most common error with the render farm, so please check all your file paths carefully. TIP: in the hypershade, use Edit>Delete Duplicate Shading Networks and Edit>Delete Unused Nodes to remove excess materials and cut down on clutter.
    • We have added a tool to make asset checking more efficient, especially if you have a large amount of textures. Go to the Deadline shelf in Maya with your scene open and click on the image icon. This will open the File Texture Manager. Click on "Analyze Scene File Textures" to generate a report of your textures. If the files are not located on the R:\ drive, you can check the boxes next to the missing files and use the buttons on the bottom of the window to copy the files to the "sourceimages" folder, and then set path as well. Refer to this video for an example of how this tool is used: File Texture Manager Video.
  • In your render settings, ensure that your output (e.g. ‘images’) is also set relative to your project folder on R:\.
  • In your render settings, ensure that you have selected an image sequence output, rather than a video file like .mov or .avi (Video renders not supported by network rendering).
  • Check that your file padding (file padding refers to the amount of zeros you put on the frame number) & extension order is appropriate: do not put the frame numbering at the end of the filename. This will break the file since it will try and read the numbers as the file extension. For Redshift users, although you cannot change the order, the default it gives you is functional.


Step Four - After making any changes ensure that you re-save the file to R.


Step Five - Submit Job from Maya using Deadline tool tab.
Deadline submission button
  • With your project open, open the Deadline tool tab and hit the Submit Job button. This will launch a separate Deadline submission dialogue that contains your job details auto-filled based on project settings & best practice defaults.
Deadline Submission Form
  • Name your job with your first initial and last name and something descriptive (cstrickler_sc01_sh03_group_scene_v02). Use v01, v02, etc... if you need to re-render a scene
  • Enter your frame list and set the camera.
  • Double check that your scene file & project directories are correct & relative to R:\. If you see C:\ or S:\ or any other letter at the beginning of the path, your render will not work.
  • Hit the "Submit" button.
Successful Submission


Step Six - Manage your job
Deadline Monitor
Once the job has been submitted, you can open the Deadline application "Monitor" to keep track of your job and about other activity on the farm. To open Monitor, you can find it in the start menu under "Network Rendering", it has a green computer monitor as an icon.
At this point you can close Maya and let the computer idle, after 15 minutes the render farm will detect the unused computer and use it for rendering.
  • From here you can keep track of your render. The left panel is a list of all the jobs currently rendering and completed. If you click on your job, the panel on the right will let you know information about each frame.
  • Within 30 seconds your job should change from ‘queued’ to ‘rendering’, unless there is a long queue on the farm.
  • Your job may gather (many) errors, but it is most likely a computer that tried to pick up your job but was unable to render because it was missing a plugin or was interrupted by someone needing to use that computer.
  • If your job does not begin after a few minutes or gathers many errors without making progress, double check your image paths and render output and make sure they all are on the R:\ drive.
  • Notice a particular computer is consistently messing up your render? You can right-click on a frame rendered by the culprit and select "Requeue Tasks by *computer name*". A dialogue box with a checkbox will come up and ask if you want to blacklist the computer. Click the checkbox and hit "OK" and all tasks previously rendered by that computer will be redone, and that computer will be prevented from rendering your job again. If this happens, please email us at animtech@ecuad.ca so we can take a look at that computer.
  • Do not delete jobs, we can use the error reports to troubleshoot what may be wrong with your render.


Step Seven - Retrieve your files
  • Once your job is completed, you can retrieve your files from the images folder of your project folder on the Render Drive.
    • If you are on PCoIP, you can transfer your image sequence over OneDrive, or simply use the PCoIP computer to create a video that you can more easily transfer to OneDrive or upload to moodle, Youtube, Vimeo, etc. . Make sure to back up your work.
  • Please delete your rendered images and project folder once you are sure you don't need them anymore. We need to keep the Render Farm clean and tidy in order to ensure it runs smoothly. All files will be deleted from the Render Drive in May.

Failed Jobs

If your render is slow or producing undesirable results, you can halt your process in one of a couple ways:
Right click:
  • Suspend - This will pause the current job, but allow frames that are currently rendering to finish.
  • Fail Job - This will pause the current job, dropping everything.
Please only manage/modify your own jobs.
Do not delete jobs, we can use the error reports to troubleshoot what may be wrong with your render.

Main Reasons Render Jobs Fail

The 5 most common network rendering errors in order of frequency
  • 1. Files and folders located on the local scratch drive rather than on the render drive. This is by far the most common error. Please check your textures to make sure they are in the right location.
  • 2. Incorrect project paths (e.g. C:\ instead of R:\).
  • 3. Missing project files, especially textures.
  • 4. Missing plugins/scripts: Emily Carr must own all the plugins you are using , or be able to download them for free! Cracked plugins will not work!
  • 5. Inefficiently or bizarrely constructed scene files.

If the job keeps failing after you’ve ruled out all these things, please email animtech@ecuad.ca