Audience
Neural networks experts looking for a solution to run deep-learning frameworks as a service on Kubernetes
About Fabric for Deep Learning (FfDL)
Deep learning frameworks such as TensorFlow, PyTorch, Caffe, Torch, Theano, and MXNet have contributed to the popularity of deep learning by reducing the effort and skills needed to design, train, and use deep learning models. Fabric for Deep Learning (FfDL, pronounced “fiddle”) provides a consistent way to run these deep-learning frameworks as a service on Kubernetes. The FfDL platform uses a microservices architecture to reduce coupling between components, keep each component simple and as stateless as possible, isolate component failures, and allow each component to be developed, tested, deployed, scaled, and upgraded independently. Leveraging the power of Kubernetes, FfDL provides a scalable, resilient, and fault-tolerant deep-learning framework. The platform uses a distribution and orchestration layer that facilitates learning from a large amount of data in a reasonable amount of time across compute nodes.
Other Popular Alternatives & Related Software
Horovod
Horovod was originally developed by Uber to make distributed deep learning fast and easy to use, bringing model training time down from days and weeks to hours and minutes. With Horovod, an existing training script can be scaled up to run on hundreds of GPUs in just a few lines of Python code. Horovod can be installed on-premise or run out-of-the-box in cloud platforms, including AWS, Azure, and Databricks. Horovod can additionally run on top of Apache Spark, making it possible to unify data processing and model training into a single pipeline. Once Horovod has been configured, the same infrastructure can be used to train models with any framework, making it easy to switch between TensorFlow, PyTorch, MXNet, and future frameworks as machine learning tech stacks continue to evolve.
Learn more about Horovod
AWS Deep Learning AMIs
AWS Deep Learning AMIs (DLAMI) provides ML practitioners and researchers with a curated and secure set of frameworks, dependencies, and tools to accelerate deep learning in the cloud. Built for Amazon Linux and Ubuntu, Amazon Machine Images (AMIs) come preconfigured with TensorFlow, PyTorch, Apache MXNet, Chainer, Microsoft Cognitive Toolkit (CNTK), Gluon, Horovod, and Keras, allowing you to quickly deploy and run these frameworks and tools at scale. Develop advanced ML models at scale to develop autonomous vehicle (AV) technology safely by validating models with millions of supported virtual tests. Accelerate the installation and configuration of AWS instances, and speed up experimentation and evaluation with up-to-date frameworks and libraries, including Hugging Face Transformers. Use advanced analytics, ML, and deep learning capabilities to identify trends and make predictions from raw, disparate health data.
Learn more about AWS Deep Learning AMIs
Deeplearning4j
DL4J takes advantage of the latest distributed computing frameworks including Apache Spark and Hadoop to accelerate training. On multi-GPUs, it is equal to Caffe in performance. The libraries are completely open-source, Apache 2.0, and maintained by the developer community and Konduit team. Deeplearning4j is written in Java and is compatible with any JVM language, such as Scala, Clojure, or Kotlin. The underlying computations are written in C, C++, and Cuda. Keras will serve as the Python API. Eclipse Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Apache Spark, DL4J brings AI to business environments for use on distributed GPUs and CPUs. There are a lot of parameters to adjust when you're training a deep-learning network. We've done our best to explain them, so that Deeplearning4j can serve as a DIY tool for Java, Scala, Clojure, and Kotlin programmers.
Learn more about Deeplearning4j
Google Cloud Deep Learning VM Image
Provision a VM quickly with everything you need to get your deep learning project started on Google Cloud. Deep Learning VM Image makes it easy and fast to instantiate a VM image containing the most popular AI frameworks on a Google Compute Engine instance without worrying about software compatibility. You can launch Compute Engine instances pre-installed with TensorFlow, PyTorch, scikit-learn, and more. You can also easily add Cloud GPU and Cloud TPU support. Deep Learning VM Image supports the most popular and latest machine learning frameworks, like TensorFlow and PyTorch. To accelerate your model training and deployment, Deep Learning VM Images are optimized with the latest NVIDIA® CUDA-X AI libraries and drivers and the Intel® Math Kernel Library. Get started immediately with all the required frameworks, libraries, and drivers pre-installed and tested for compatibility. Deep Learning VM Image delivers a seamless notebook experience with integrated support for JupyterLab.
Learn more about Google Cloud Deep Learning VM Image
Company Information
IBM
Founded: 1911
United States
developer.ibm.com/open/projects/fabric-for-deep-learning-ffdl/
Videos and Screen Captures
Other Useful Business Software
Cut Data Warehouse Costs by 54%
BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
Product Details
Platforms Supported
Cloud
Training
Documentation
Webinars
Videos
Support
Online
Fabric for Deep Learning (FfDL) Frequently Asked Questions
Q: What kinds of users and organization types does Fabric for Deep Learning (FfDL) work with?
Fabric for Deep Learning (FfDL) works with these users and organization types: Mid Size Business, Small Business, Enterprise, Freelance, Nonprofit, Government, and Startup.
Q: What languages does Fabric for Deep Learning (FfDL) support in their product?
Fabric for Deep Learning (FfDL) supports these languages: English.
Q: What kind of support options does Fabric for Deep Learning (FfDL) offer?
Fabric for Deep Learning (FfDL) offers support via online.
Q: What other applications or services does Fabric for Deep Learning (FfDL) integrate with?
Fabric for Deep Learning (FfDL) integrates with: Caffe, Kubernetes, PyTorch, TensorFlow, and Torch.
Q: Does Fabric for Deep Learning (FfDL) have an API?
Yes, Fabric for Deep Learning (FfDL) has an API.
Q: What type of training does Fabric for Deep Learning (FfDL) provide?
Fabric for Deep Learning (FfDL) provides training in the form of documentation, webinars, and videos.
