Tag Archives: 機器學習

[ 翻譯 ] 2018.12.07- 部署機器學習模型更簡單!!如何使用TensorFlow Serving搭配Docker?






Designed by Freepik


Gautam Vasudevan與Abhijit Karmarkar,特此致謝!






Serving machine learning models quickly and easily is one of the key challenges when moving from experimentation into production. Serving machine learning models is the process of taking a trained model and making it available to serve prediction requests. When serving in production, you want to make sure your environment is reproducible, enforces isolation, and is secure. To this end, one of the easiest ways to serve machine learning models is by using TensorFlow Serving with Docker. Docker is a tool that packages software into units called containers that include everything needed to run the software.

當從實驗階段移至生產階段,如何快速、簡單地部署機器學習模型,是其中一項關鍵性挑戰。部署機器學習模型,是取用一個訓練好的模型並使此模型能回應預測請求的過程。當把模型部署至實際生產時,使用者想要確保環境是可重現、獨立且安全的。至此,部署機器學習模型最簡單的方法之一,是使用TensorFlow Serving搭配Docker。什麼是Docker呢?Docker是一項把軟體打包成一個個單元的工具,而這樣的單元被稱作「容器」,它包含了運作該軟體的一切所需。


Since the release of TensorFlow Serving 1.8, we’ve been improving our support for Docker. We now provide Docker images for serving and development for both CPU and GPU models. To get a sense of how easy it is to deploy a model using TensorFlow Serving, let’s try putting the ResNet model into production. This model is trained on the ImageNet dataset and takes a JPEG image as input and returns the classification category of the image.

自TensorFlow Serving 1.8發佈以來,我們持續改善對Docker的支援。現在,我們提供了Docker 映像檔,讓使用者可針對CPU與GPU模型進行部署和開發。為讓讀者們了解,運用TensorFlow Serving部署模型到底有多簡單,讓我們試著使ResNet模型進行生產。這個模型是以ImageNet資料集來訓練,以JPEG圖像作為輸入,並會回傳此圖像的分類結果。


Our example will assume you’re running Linux, but it should work with little to no modification on macOS or Windows as well.



Serving ResNet with TensorFlow Serving and Docker

The first step is to install Docker CE. This will provide you all the tools you need to run and manage Docker containers.

TensorFlow Serving uses the SavedModel format for its ML models. A SavedModel is a language-neutral, recoverable, hermetic serialization format that enables higher-level systems and tools to produce, consume, and transform TensorFlow models. There are several ways to export a SavedModel(including from Keras). For this exercise, we will simply download a pre-trained ResNet SavedModel:

運用TensorFlow Serving與Docker部署ResNet

第一步是安裝Docker CE,它將提供運作並管理Docker容器所需的工具。

針對其下各種機器學習模型,TensorFlow Serving使用SavedModel格式。SavedModel是一種語言中立、可回復、密閉序列化的格式,使高階系統和工具得以產生、運用並轉化TensorFlow模型。匯出SavedModel格式(包括Keras的模型)的方法相當多元,本範例將下載預先訓練好的ResNet SavedModel。

$ mkdir /tmp/resnet
$ curl -s https://storage.googleapis.com/download.tensorflow.org/models/official/20181001_resnet/savedmodels/resnet_v2_fp32_savedmodel_NHWC_jpg.tar.gz | tar --strip-components=2 -C /tmp/resnet -xvz


We should now have a folder inside /tmp/resnet that has our model. We can verify this by running:


$ ls /tmp/resnet


Now that we have our model, serving it with Docker is as easy as pulling the latest released TensorFlow Serving serving environment image, and pointing it to the model:

有了模型之後,要運用Docker部署模型就簡單了,只要使用pull指令取得最新發佈的TensorFlow Serving的serving environment映像檔,並且將serving environment映像檔指向模型即可:

$ docker pull tensorflow/serving
$ docker run -p 8501:8501 --name tfserving_resnet \
--mount type=bind,source=/tmp/resnet,target=/models/resnet \
-e MODEL_NAME=resnet -t tensorflow/serving &
… main.cc:327] Running ModelServer at…
… main.cc:337] Exporting HTTP/REST API at:localhost:8501 …


Breaking down the command line arguments, we are:


  • -p 8501:8501: Publishing the container’s port 8501 (where TF Serving responds to REST API requests) to the host’s port 8501

發佈容器埠8501(TensorFlow Serving在此回應REST API請求)對應到主機埠8501。


  • –name tfserving_resnet: Giving the container we are creating the name “tfserving_resnet” so we can refer to it later



  • –mount type=bind,source=/tmp/resnet,target=/models/resnet: Mounting the host’s local directory (/tmp/resnet) on the container (/models/resnet) so TF Serving can read the model from inside the container.

運用mount命令,將主機的本地目錄(/tmp/resnet)掛載至容器上(/models/resnet)。這樣,TensorFlow Serving可從容器內讀取模型。


  • -e MODEL_NAME=resnet: Telling TensorFlow Serving to load the model named “resnet”

告訴TensorFlow Serving,載入名稱為「resnet」的模型。


  • -t tensorflow/serving: Running a Docker container based on the serving image “tensorflow/serving”



Next, let’s download the python client script, which will send the served model images and get back predictions. We will also measure server response times.


$ curl -o /tmp/resnet/resnet_client.py https://raw.githubusercontent.com/tensorflow/serving/master/tensorflow_serving/example/resnet_client.py


This script will download an image of a cat and send it to the server repeatedly while measuring response times, as seen in the main loop of the script:


# The server URL specifies the endpoint of your server running the ResNet
# model with the name "resnet" and using the predict interface.
SERVER_URL = 'http://localhost:8501/v1/models/resnet:predict'


# Send few actual requests and time average latency.                                                                                                                                                                   
total_time = 0
num_requests = 10
for _ in xrange(num_requests):
    response = requests.post(SERVER_URL, data=predict_request)
total_time += response.elapsed.total_seconds()
prediction = response.json()['predictions'][0]

print('Prediction class: {}, avg latency: {} ms'.format(
prediction['classes'], (total_time*1000)/num_requests))


This script uses the requests module, so you’ll need to install it if you haven’t already. By running this script, you should see output that looks like:


$ python /tmp/resnet/resnet_client.py
Prediction class: 282, avg latency: 185.644 ms


As you can see, bringing up a model using TensorFlow Serving and Docker is pretty straight forward. You can even create your own custom Docker imagethat has your model embedded, for even easier deployment.

從上面的實例可知,運用TensorFlow Serving與Docker部署模型十分直接。讀者甚至可以建置自己的客製Docker映像檔,其中內嵌您的模型,部署起來更加容易。


Improving performance by building an optimized serving binary

Now that we have a model being served in Docker, you may have noticed a log message from TensorFlow Serving that looks like:


既然我們已將模型部署至Docker中,讀者可能已經注意到一則來自TensorFlow Serving的log訊息,如下:

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA


The published Docker images for TensorFlow Serving are intended to work on as many CPU architectures as possible, and so some optimizations are left out to maximize compatibility. If you don’t see this message, your binary is likely already optimized for your CPU.

針對TensorFlow Serving已發佈的Docker映像檔,其目的是盡可能在更多CPU架構上運作,所以會放棄某些最佳化措施來提高相容性。如果您並未看見上述訊息,代表您的二元檔案可能已針對您的CPU進行了最佳化。


Depending on the operations your model performs, these optimizations may have a significant effect on your serving performance. Thankfully, putting together your own optimized serving image is straightforward.



First, we’ll want to build an optimized version of TensorFlow Serving. The easiest way to do this is to build the official Tensorflow Serving development environment Docker image. This has the nice property of automatically generating an optimized TensorFlow Serving binary for the system the image is building on. To distinguish our created images from the official images, we’ll be prepending $USER/ to the image names. Let’s call this development image we’re building $USER/tensorflow-serving-devel:

首先,我們要建置一個TensorFlow Serving經過最佳化的版本。最簡單的方式,是建置官方的TensorFlow Serving開發環境的Docker映像檔。這個映像檔有個很棒的屬性,就是可針對映像檔立基的系統,自動產生一個經過最佳化的TensorFlow二元檔。為區分我們自己建置的映像檔和官方的映像檔,我們將在自己建置的映像檔名稱前加上「$USER/」,讓我們把它命名為「$USER/tensorflow-serving-devel」:

$ docker build -t $USER/tensorflow-serving-devel \
-f Dockerfile.devel \ 


Building the TensorFlow Serving development image may take a while, depending on the speed of your machine. Once it’s done, let’s build a new serving image with our optimized binary and call it $USER/tensorflow-serving:

建置自己的TensorFlow Serving開發映像檔根據您的電腦規格可能會花上一段時間。一旦建置完成,就能運用最佳化後的二元檔來建置一個新的部署映像檔,並將它命名為「$USER/tensorflow-serving」:

$ docker build -t $USER/tensorflow-serving \
--build-arg TF_SERVING_BUILD_IMAGE=$USER/tensorflow-serving-devel \ https://github.com/tensorflow/serving.git#:tensorflow_serving/tools/docker


Now that we have our new serving image, let’s start the server again:


$ docker kill tfserving_resnet
$ docker run -p 8501:8501 --name tfserving_resnet \
  --mount type=bind,source=/tmp/resnet,target=/models/resnet \
  -e MODEL_NAME=resnet -t $USER/tensorflow-serving &


And finally run our client:


$ python /tmp/resnet/resnet_client.py
Prediction class: 282, avg latency: 84.8849 ms


On our machine, we saw a speedup of over 100ms (119%) on average per prediction with our native optimized binary. Depending on your machine (and model), you may see different results.



Finally, feel free to kill the TensorFlow Serving container:

最後,請用「kill」指令來終止TensorFlow Serving容器:

$ docker kill tfserving_resnet


Now that you have TensorFlow Serving running with Docker, you can deploy your machine learning models in containers easily while maximizing ease of deployment and performance.

既然,您已讓TensorFlow Serving與Docker一同運作,自然能輕鬆地部署機器學習模型至容器內,不但能輕鬆部署,效能也能最大化。


Please read our Using TensorFlow Serving via Docker documentation for more details, and star our GitHub project to stay up to date.








[ 翻譯 ] 2018.12.3- TensorFlow Lite專案分享:Jrobot無人四輪車




Joe Chen、CAVEDU教育團隊


Joe Chen

感謝作者Joe Chen授權翻譯文字內容,以及播放影片,特此致謝!


Designed by Freepik,特此致謝!


Jrobot Self-Drive Powered by TensorFlow Lite

Jrobot無人四輪車的運作核心是TensorFlow Lite。


Previously, we used J-control to drive the Jrobot remotely, and this time we get rid of J-control and let Jrobot to do self-drive.



Jrobot Self Drive is another self-drive experiment based on machine learning, it is not a simulator, it is not a road vehicle, it is a footpath traveler. We built NVIDIA CNN self-drive model using Keras, collected training data, trained the model, and converted the trained model to TensorFlow Lite.

Jrobot無人四輪車是一項建基於機器學習的無人車實驗。Jrobot不是模擬器,也不是道路用車輛,而是一台小型的四輪車。我們運用Keras建構NVIDIA CNN自駕模型,然後收集訓練資料,訓練模型,最後將訓練好的模型轉換至TensorFlow Lite上。


TensorFlow Lite allows us to do inference on a mobile device and is the key part of this project. We added TensorFlow Lite to Jrobot Android app. When running, TensorFlow Lite is able to load the trained model, take a camera image as input and give a steering angle as output. Jrobot app runs on an Android phone (Xiaomi Mi5) sitting in the phone box on Jrobot car and control the movement of the Jrobot car through bluetooth connection with Arduino on the car.

運用TensorFlow Lite使我們可以在行動裝置上進行推論,所以TensorFlow Lite可說是這個專案的核心。我們把TensorFlow Lite加至Jrobot Android app中,當運作時,TensorFlow Lite能夠載入訓練好的模型,然後以照相機的圖像作輸入,並給出一個轉向角作輸出。Jrobot app在Android手機上運作,在下面的影片中,我們使用的手機是小米Mi5。當Jrobot準備行駛前,我們把手機放入Jrobot無人四輪車上的手機盒,並透過手機上的藍牙裝置連接四輪車上的Arduino開發板,以控制四輪車的行駛。


We did road test in 2 places in the neighborhood and the results show us the trained model works well. Even though it is not full self-drive, it makes human control so much easier, and opens up so many new options, which means there is so much more to do. Thank you!









[ 介紹文 ] 2018.10.05- 人工智慧大師Andrew Ng的英文新書Machine Learning Yearning,草稿搶先讀!




截圖自「Machine Learning Yearning」網站,特此致謝!



AI人工智慧大師Andrew Ng(相關簡介)的英文新書Machine Learning Yearning,草稿已全部完成。以下是他對這本新著作的分享:


Machine Learning Yearning最後幾章草稿完成了!藉由這幾章的內容,您將可學到機器學習資料流(ML pipelines)的錯誤分析(Error Analysis)。舉例來說,如果您有個複雜的系統,比如一輛自駕車,它包含許多子元件。在這樣情況下,您如何決定使用哪個元件?因此,我盼望透過這本書的最後幾章,幫助您與您的團隊,更有效建構各種機器學習專案!


目前,本書仍處於草稿階段,為讓它對您與您的團隊更有效益,我非常歡迎任何回饋。請將建議寄我們:hello@deeplearning.ai 。」








[ 翻譯 ] 2018.09.21- 認識TensorFlow.js,在瀏覽器就能用JavaScript 訓練機器學習模型喔!


Josh Gordon  Sara Robinson


Josh Gordon、Sara Robinson提供



We’re excited to introduce TensorFlow.js, an open-source library you can use to define, train, and run machine learning models entirely in the browser, using Javascript and a high-level layers API. If you’re a Javascript developer who’s new to ML, TensorFlow.js is a great way to begin learning. Or, if you’re a ML developer who’s new to Javascript, read on to learn more about new opportunities for in-browser ML. In this post, we’ll give you a quick overview of TensorFlow.js, and getting started resources you can use to try it out.




In-Browser ML

Running machine learning programs entirely client-side in the browser unlocks new opportunities, like interactive ML! If you’re watching the livestream for the TensorFlow Developer Summit, during the TensorFlow.js talk you’ll find a demo where @dsmilkov and @nsthorat train a model to control a PAC-MAN game using computer vision and a webcam, entirely in the browser. You can try it out yourself, too, with the link below — and find the source in the examples folder.



在瀏覽器客戶端就能直接執行機器學習程式能帶出許多新契機,例如互動式的機器學習!您可以看看TensorFlow開發者大會直播,在關於TensorFlow.js的分享中有一段是@dsmilkov@nsthorat 正demo一個完全跑在瀏覽器中的範例,運用電腦視覺和網路攝影機訓練一個模型以控制小精靈遊戲。您也可以使用下面的連結親自試一下,並在examples資料夾中找到原始程式碼。


If you’d like to try another game, give the Emoji Scavenger Hunt a whirl — this time, from a browser on your mobile phone.


如果想試試看其他遊戲,可以試試看Emoji Scavenger Hunt ,但這次請使用您手機上的瀏覽器嘗試。



ML running in the browser means that from a user’s perspective, there’s no need to install any libraries or drivers. Just open a webpage, and your program is ready to run. In addition, it’s ready to run with GPU acceleration. TensorFlow.js automatically supports WebGL, and will accelerate your code behind the scenes when a GPU is available. Users may also open your webpage from a mobile device, in which case your model can take advantage of sensor data, say from a gyroscope or accelerometer. Finally, all data stays on the client, making TensorFlow.js useful for low-latency inference, as well as for privacy preserving applications.




What can you do with TensorFlow.js?

If you’re developing with TensorFlow.js, here are three workflows you can consider.

  • You can import an existing, pre-trained model for inference. If you have an existing TensorFlow or Kerasmodel you’ve previously trained offline, you can convert into TensorFlow.js format, and load it into the browser for inference.


  • You can re-train an imported model. As in the Pac-Man demo above, you can use transfer learning to augment an existing model trained offline using a small amount of data collected in the browser using a technique called Image Retraining. This is one way to train an accurate model quickly, using only a small amount of data.


  • Author models directly in browser. You can also use TensorFlow.js to define, train, and run models entirely in the browser using Javascript and a high-level layers API. If you’re familiar with Keras, the high-level layers API should feel familiar.




  • 您可匯入現有、預先訓練好的模型進行推論。若您有一個現有的TensorFlow或Keras模型,您可將其轉換為js的檔案格式,並將其載入至瀏覽器中進行推論。


  • 您可以重新訓練一個已匯入的模型。在前面提及的小精靈遊戲範例中,您可使用遷移式學習,透過使用一種叫做「影像再訓練(Image Retraining)」的技術,運用在瀏覽器中收集到的少量數據,做到在離線狀態下也能訓練現有模型。這是一種只需少量資料也能快速訓練精確模型的方法。


  • 直接在瀏覽器中建立模型。您也可以用TensorFlow.js,運用Javascript和高階API,直接在瀏覽器內定義、訓練並運作模型。若您熟悉Keras,那麼這些高層API對您來說應該不陌生。


Let’s see some code

If you like, you can head directly to the samples or tutorials to get started. These show how-to export a model defined in Python for inference in the browser, as well as how to define and train models entirely in Javascript. As a quick preview, here’s a snippet of code that defines a neural network to classify flowers, much like on the getting started guide on TensorFlow.org. Here, we’ll define a model using a stack of layers.




import * as tf from ‘@tensorflow/tfjs’;
const model = tf.sequential();
model.add(tf.layers.dense({inputShape: [4], units: 100}));
model.add(tf.layers.dense({units: 4}));
model.compile({loss: ‘categoricalCrossentropy’, optimizer: ‘sgd’});


The layers API we’re using here supports all of the Keras layers found in the examples directory (including Dense, CNN, LSTM, and so on). We can then train our model using the same Keras-compatible API with a method call:


這個layers API支援範例目錄中所有的Keras層(包括Dense、CNN、LSTM等等)。接著,只要呼叫對應的方法就可以使用相容於Keras的API訓練模型。以下是這個方法:

await model.fit(
  xData, yData, {
    batchSize: batchSize,
    epochs: epochs


The model is now ready to use to make predictions:


// Get measurements for a new flower to generate a prediction
// The first argument is the data, and the second is the shape.
const inputData = tf.tensor2d([[4.8, 3.0, 1.4, 0.1]], [1, 4]);

// Get the highest confidence prediction from our model
const result = model.predict(inputData);
const winner = irisClasses[result.argMax().dataSync()[0]];

// Display the winner


TensorFlow.js also includes a low-level API (previously deeplearn.js) and support for Eager execution. You can learn more about these by watching the talk at the TensorFlow Developer Summit.


TensorFlow.js也包括一個低階的API(前身是deeplearn.js),並且支援Eager execution。請參考TensorFlow開發者大會的內容,您就能更深入了解。

圖2 TensorFlow.js API的概述


How does TensorFlow.js relate to deeplearn.js?

Good question! TensorFlow.js, an ecosystem of JavaScript tools for machine learning, is the successor to deeplearn.js which is now called TensorFlow.js Core. TensorFlow.js also includes a Layers API, which is a higher level library for building machine learning models that uses Core, as well as tools for automatically porting TensorFlow SavedModels and Keras hdf5 models. For answers to more questions like this, check out the FAQ.



這是一個好問題!TensorFlow.js是一個為了機器學習而誕生的JavaScript工具生態系,是deeplearn.js的繼承者,而deeplearn.js現在則被稱為TensorFlow.js Core。TensorFlow.js還包括一個Layers API,這是一個更高階的函式庫,能運用Core來建置機器學習模型,它也是一款自動移植TensorFlow SavedModels與Keras hdf5模型的工具。更多類似問題的答案,請參閱FAQ


Where’s the best place to learn more?

To learn more about TensorFlow.js, visit the project homepage, check out the tutorials, and try the examples. You can also watch the talk from the 2018 TensorFlow Developer Summit, and follow TensorFlow on Twitter.





Thanks for reading, and we’re excited to see what you’ll create with TensorFlow.js! If you like, you can follow @dsmilkov@nsthorat, and @sqcaifrom the TensorFlow.js team on Twitter for updates.


謝謝您耐心讀完本文,我們很期待看到您運用TensorFlow.js做出的成果!喜歡的話,可以在Twitter上關注TensorFlow.js團隊的@dsmilkov@nsthorat@ sqcai 等人的帳號來獲得最新消息。





[課程紀錄文] 2018.8.13-14-跟著CAVEDU學AI人工智慧:一堂結合機器學習中的影像辨識及邊緣運算的深度課程











然而,早在課程開始前的一個月,CAVEDU教育團隊這邊的兩位講師:「阿吉老師」曾吉弘與徐豐智,以及與此次課程的相關人員,便已針對這門課程展開緊鑼密鼓地準備。從剛開始針對課程內容進行深入研究、討論;至中期的針對「Microsoft Azure DSVM資料科學虛擬機器」持續調整、設定,並且訓練模型;最後一週與課程各單位間的持續溝通、協調、聯繫,針對上課要使用的RK Car不斷調測,設計、印刷試跑RK Car需用的道路地墊⋯⋯等等。可以說,CAVEDU教育團隊真是為這次課程卯足全勁了!


課程的結果也是十分甜美。根據微軟方面人員的統計,參與課程21位學員的AI AGV車皆成功跑車,一個步驟都沒有漏掉。而微軟方面亦十分肯定CAVEDU教育團隊,在這次課程中展現出的專業能力

圖1 講師之一「阿吉老師」曾吉弘


圖2 另外一位講師徐豐智(中著紅衣者)





圖3 學員讓RK Car在CAVEDU製作的道路地墊上試跑


我們運用什麼工具訓練模型呢?答案是微軟的Azure DSVM資料科學虛擬機器。Azure DSVM是一種雲端服務,透過常用於資料分析、機器學習服務及AI訓練的數個熱門工具,預先安裝、設定及測試的Azure虛擬機器映像。

圖4 資料蒐集並訓練模型


下圖是課程中使用的無人載具RK Car,車子由CAVEDU教育團隊自造,以Raspberry Pi單板電腦為核心。RK Car上藍色的是Intel Movidius神經運算棒,阿吉老師有向學員們展示Intel的AI模型,透過Movidius的加速,針對道路上各種車輛進行分類,以及街景影像的辨識,例如街道上的人、車牌、摩托車⋯⋯等等。透過Intel Movidius,能使硬體等級沒有非常高的Raspberry Pi單板電腦,以更快速度進行影像辨識。(若欲購買RK Car,請洽機器人王國商城。)

圖5 安裝上Intel Movidius神經運算棒的RK Car



圖6 所有學員順利完成跑車