LinkedIn has just open sourced a tool it created to convert Apache Spark-based big data into a format that can be readily consumed by TensorFlow. The tool, Avro2TF “bridges the gap and presents an elegant solution for ML engineers, freeing them up to focus on different deep learning algorithms,” the team stated.

The team noted that like LinkedIn, many other companies have vast amounts of ML data in a similar sparse vector format, and Tensor format is still relatively new to many of them. The believe that Avro2TF can help address these challenges.

It’s interesting to see LinkedIn, which itself is owned by Microsoft, open source their internal AI/ML tooling, joining the likes of Uber, Facebook, etc. What a time to be alive! 🙂

tt ads

One comment

  • Sankar

    Great news! This should tremendously save the time for deep learning model developers who are using big data for their analysis.

Comments are closed.