SpaceFarmers.io Wiki

Automated plotting on Linux with Plotman

Reference Sites

Install on plotting machine
- https://chiadecentral.com/plotman-automate-and-manage-chia-plotting/
Configure archiver to move plots to external or remote drives
- https://github.com/ericaltendorf/plotman/wiki/Archiving

Installing Plotman

Ensure that you have at least Python 3.7 installed and Pip installed
- For help go to https://packaging.python.org/tutorials/installing-packages/
Activate your Chia environment
- . ./activate
Install Plotman
- pip install –force-reinstall git+https://github.com/ericaltendorf/plotman@main
Verify that Plotman is working with
- plotman status

Configure Plotting and Destination Drives

Create a default plotman.yaml config file
- plotman config generate
- This will be located in ~/.config/plotman/plotman.yaml
Plotman is designed to maximize the number of plots running in parallel
- You need to tell it which drives you use for
  - plotting (primary and secondary)
  - destination
  - (optionally) archiving
- And how to stagger jobs and the amount of resources to use
- An example config for plotting with two SSD's, a second plotting temp drive, and a single destination drive
  - ```
  directories:
     # normally NVMe SSD or SAS
     tmp:
        - /data/s970evop_1
        - /data/mp510_1
  
     # only worthwhile if NVMe or SATA SSD
     tmp2: /data/crp1_1/tmp
  
     # normally spinning rust
     dst:
        - /data/wdredp_1
```

Configure Job Staggering

An example config that allows 6 parallel jobs in total, with no more than 3 jobs per primary plotting drive, but only 2 jobs per drive in phase 1

scheduling:
   tmpdir_stagger_phase_major: 2
   tmpdir_stagger_phase_minor: 1
   tmpdir_stagger_phase_limit: 2   # no more than 2 jobs per drive for above phase thresholds
   tmpdir_max_jobs: 3
   global_max_jobs: 6

Configure Plotman Plotting for Pools

To allow creation of plots for a pool, add the pool contract address to the config file
- ```
plotting:
   pool_contract_address: xch..............
```
- Find your pool contract address with the command
  - chia plotnft show

Configure Chia Plot Parameters

Example for rig with 12 hyperthreads, 32 GB RAM, and max of 6 parallel jobs

plotting:
type: chia
chia:
  k: 32                # k-size of plot, leave at 32 most of the time
  e: False             # Use -e plotting option
  n_threads: 4         # Threads per job: Can overcommit providing no more than 4 jobs allowed in phase 1
  n_buckets: 128       # Number of buckets to split data into
  job_buffer: 5120     # Per job memory: Increase above default 3390 if RAM to spare for marginal improvement

Configure Archiving

Plotman allows optional archiving to either local or remote drives

Although rsyncd is recommended for remote drives, it is also possible to use remote NFS drives and Plotman will think that they are local drives

An example config for archiving with throttled bandwidth to a harvester running on a Raspberry Pi with NFS drives is

archiving:
  target: rsync_rsync_throttle
  env:
    site_root: /nfs/chia
  target_definitions:
    rsync_rsync_throttle:
      env:
        command: rsync
        # adjust bandwidth limit in bytes/second to avoid overloading device hosting archive drives
        options: --bwlimit=35000 --preallocate --remove-source-files --skip-compress plot --whole-file
        site_root: null
      disk_space_script: |
        #!/bin/bash
        set -evx
        df -BK | grep " ${site_root}/" | awk '{ gsub(/K$/,"",$4); printf "%s:%.0f\n", $6, $4*1024 }'
      transfer_script: |
        #!/bin/bash
        set -evx
        "${command}" ${options} "${source}" "${destination}"
      transfer_process_name: "{command}"
      transfer_process_argument_prefix: "{site_root}"

Running Plotman

From the CLI the three commands you will use the most are

plotman status
- Gives an overview of all the jobs running
plotman interactive
- Interactive screen showing jobs and archiving progress
- Works well with headless systems, i.e. SSH friendly
- Stop and start plotting and archiving using the <p> and <a> keys
- Quit the interactive screen by pressing <q>

plotman analyze

Allows you to see how many jobs you have created over period
For example:
plotman analyze ~/.chia/mainnet/log/*2021-08-09*

+-------+----+-------------+---------------+--------------+---------------+---------------+--------------+
| Slice | n  |   %usort    |    phase 1    |   phase 2    |    phase 3    |    phase 4    |  total time  |
+=======+====+=============+===============+==============+===============+===============+==============+
| x     | 18 | μ=100.0 σ=0 | μ=12.6K σ=659 | μ=5.4K σ=307 | μ=11.6K σ=817 | μ=854.6 σ=132 | μ=30.5K σ=1K |
+-------+----+-------------+---------------+--------------+---------------+---------------+--------------+

Table of Contents