Miscellaneous server-side improvements

[justheuristic]

• task_pool

  • remove interate_minibatches, once Require TaskPoolBase to implement load_batch_to_runtime learning-at-home/hivemind#506 is merged
  • batch sequences of similar length

• runtime

  • do not log "0 parameters" on init (misleading)
  • consider removing Runtime, once Change runtime.py to choose tasks with lowest (instead of highest) priority learning-at-home/hivemind#505 is merged

• handler

  • verify what happens if server have mixed torch_dtype and/or compression
  • actually follow forward/backward/inference schema instead of hard-coding
  • extract the code for adding prompts into a separate file
  • consider merging the code from hivemind's ConnectionHandler instead of inheriting
  • add a test that covers _rpc_inference with prompts
  • add a test that covers _rpc_inference with hypo-ids

• MemoryCache

  • when running inference over multiple layers on the same server, avoid passing layer activations between cpu<->gpu by
    storing them in MemoryCache
    • before implementing this, gotta check if this will bring any performance benefit
  • LRU-offload stale cache from gpu to ram

• point system

  • make sure points are integers everywhere
  • implement a nonzero prioritizer :)
  • move client-side spending polity to sequence_manager