Hi everyone,
I was wondering if anybody came across a good/simple way to do predictions from a TF model without using the TF library which seems way overkill for my simple use case. I found some leads but it’s usually an outdated code that is not compatible with latest versions of TF.
I tried TFLite, it worked but couldn’t compiled a universal binary working for Intel 32/64 & arm64 after hours of researches…
Thanks!
You can compile universal binaries for tflite bit. I’m not sure it’s worth supporting Intel 32-bit imo 
Regardless, you either need to modify the tflite dependencies to support universal binaries, or compile each architecture separately and merge the libraries into a universal architecture library. For the 1st solution I can send you a cmake script that manages that for intel/arm 64.
Hi, thanks for your answer!
I’m in the process of extracting weights and biases from the model to do the predictions with eigen…
But yes, it would be amazing if you could share your cmake!!
Here it is tflite-cmake. This is for tensorflow 2.13.0. You just have to include tflite.cmake.
Thanks a lot for your help!
Don’t know if you’ve already seen it but RTNeural was designed for fast, real-time safe inference: GitHub - jatinchowdhury18/RTNeural: Real-time neural network inferencing
It supports only a limited amount of operations, and you’ll have to extract the weights from the original TFLite model manually, but it’s nicer than reimplementing the operations with Eigen.
I didn’t look deep enough apparently, this would have been helpful… I already reimplemented the operations with Eigen and get the same results as Tensorflow with OK performances, so that will do for this one project. But thank you for pointing out to RTNeural, I’ll use it next time!
