distributed

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A hyperparameter optimization framework

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

Modin: Scale your Pandas workflows by changing a single line of code

Redis-based components for Scrapy.

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Fast job queuing and RPC in python with asyncio and redis.

Lingvo

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Official Implementation of 'Fast AutoAugment' in PyTorch.

MLBox is a powerful Automated Machine Learning python library.

[unmaintained] An open-source convolutional neural networks platform for research in medical image analysis and image-guided therapy

Examples of programs built using Modal

Redis for humans. 🌎🌍🌏

Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

Making data lake work for time series

Advanced evolutionary computation library built directly on top of PyTorch, created at NNAISENSE.

TensorFlow code for the neural network presented in the paper: "code2vec: Learning Distributed Representations of Code"