In the previous articles in this series, we have established that the common approach to AI needs rethinking, and that AI is going to hit a wall soon or already has.
To summarize, 80% of AI projects fail, and some of the dream goals of AI may remain unachievable.
Now for some positive note. You do not need to throw your AI certificates and diplomas out of the window. You just need to realize that all the AI knowledge amounts to only one-third of the complete solution. What else is needed?
- Become more practical. Know where your data is, what it means, and find out early whether the goal you plan is at all achievable with your approach. It could be that simpler models are quite enough for your project, but that it may still fail if you don’t engineer it right.
- In the Machine Learning department, don’t overengineer but make it practical. Perhaps simpler data analytics will do as well, but do you know how your AI will work in the real world? Can you draw a Machine Learning Pipeline for your project? Can you implement it? Are you using Python Notebooks that you used in all your classes? Don’t! In production, you need another approach.
- Are you a DevOps? How will you scale your model? How will you monitor it? Do you know where your customer is? Are you building your solution in the cloud? If no, why not? If yes, what is your infrastructure?
Now, for the answers. We suggest a practical approach to AI project training, which becomes more like a consultation. Here is what you will need to do.
- Figure out the goals for your team. It will be very different depending on the outcome you are seeking. The technologies will change, the approach will change.
- Decide on the correct solution pipeline and on its implementation. Bring in enough expertise in the supporting areas and know the probability of success. The engineering effort should be 3 times bigger than the Machine Learning work you do.
- Implement for the production, with mature and robust architecture, monitoring, performance testing, and enough DevOps knowledge. Keep in mind that the best AI projects never make it into production. It’s the well-engineered ones that do.
We at Elephant Scale have been thinking of this, doing this, and teaching this for years. We are running the “Machine Learning Engineering” series now. You can take our advice or we can help you implement it with our training.
In either case, we think that the more people start doing this, the better it will be for AI implementations overall. We do not promise that it will be easy: you will need more than an AI Bootcamp. You will need a lot of computer science and DevOps. But believe us: if you do good work, you will succeed.
