趁十一国庆时间充沛、可以系统给自己充充电了,认真阅读了李佳璇的《TensorFlow技术解析与实战》,该书以TensorFlow为核心,从环境的准备,到PlayGroud/TensorBoard可视化面板的展现;从编程模型、常用API的初探,到一个一个案例的实战;从神经元函数及游湖方法,到升级网络的发展及TensorFlow实现做了一一的归纳和总结,最后以MNIST数据集的应用、人脸识别的应用、自然语言处理应用、对抗网络的应用、Debugger/Kubernetes/OnSpark使用、以及训练模型在IOS/Android实战,从广度来说,对初学者来说是一本很好的读物。
知识点1:论文和实践结合学习深度学习的方法
知识点2:TensorFlow系统架构
知识点3:TensorFlow编程模型
参考地址:https://www.tensorflow.org/images/tensors_flowing.gif
知识点4:中国科学院计算技术研究所刘昕博士整理的卷积神经网络结构演化的历史
知识点5:循环神经网络发展史
知识点6:TensorFlowOnSpark系统架构图
知识点7:参考资料
参考网址(Blue):
http://yahoohadoop.tumblr.com/post/157196317141/open-sourcing-tensorflowonspark-distributed-deep
Models built with TensorFlow(其中包含official models/research models/tutorial models):
https://github.com/tensorflow/models/tree/master/
The Unreasonable Effectiveness of Recurrent Neural Networks:
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
TensorFlow官网:
Ipython官网:
http://ipython.org/notebook.html
Deep Leaning with Dynamic Computation Graphs
https://openreview.net/pdf?id=ryrGawqex
Tree-Structured Long Short-Term Memory Networks:
https://github.com/stanfordnlp/treelstm
ImageNet Classification with Deep Convolutional Neural Networks:
http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification:
https://arxiv.org/abs/1502.01852
Deep Residual Learning for Image Recognition:https://arxiv.org/abs/1512.03385
Kaggle官网:https://www.kaggle.com/
天池大数据比赛官网:https://tianchi.aliyun.com/
Yi产品:http://www.dress-plus.com/product_page1.html
Face++旷视:https://www.faceplusplus.com.cn/
讯飞开放平台:http://www.xfyun.cn/
TensorFlow GitHub:https://github.com/tensorflow/tensorflow/
PyPI - the Python Package Index:https://pypi.python.org/pypi
Image processing in Python:http://scikit-image.org/
Natural Language Toolkit:http://www.nltk.org/
Tinker With a Neural Network Righ:http://playground.tensorflow.org
Data Compression Programs:http://mattmahoney.net/dc/
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
http://download.tensorflow.org/paper/whitepaper2015.pdf
基于java语言的API(package tensorflow):
https://www.tensorflow.org/api_docs/java/reference/org/tensorflow/package-summary
基于Go语言的API(package tensorflow):
https://godoc.org/github.com/tensorflow/tensorflow/tensorflow/go
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
https://arxiv.org/abs/1502.03167
Neural Networks for Machine Learning(Coursera课程)
https://www.coursera.org/learn/neural-networks
An overview of gradient descent optimization algorithms(blue):
http://sebastianruder.com/optimizing-gradient-descent/
Tornado Web Server: http://www.tornadoweb.org/en/stable/
Notes on Convolutional Neural Networks:
http://cogprints.org/5869/1/cnn_tutorial.pdf
优达学城(UDACITY)_视频课程网:https://cn.udacity.com/courses/all
机器学习到深度学习系统系列课程(视频+项目):
https://classroom.udacity.com/courses/ud730
Gradient-Based Learning Applied to Document Recognition:
http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf
卷积可视化在线演示(blue):
https://graphics.stanford.edu/courses/cs178/applets/convolution.html
GradientBased Learning Applied to Document Recognition(blue):
http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf
ImageNet Classification with Deep Convolutional Neural Networks
Very Deep Convolutional Networks for Large-Scale Visual Recognition
http://www.robots.ox.ac.uk/~vgg/research/very_deep/
Network In Network:https://arxiv.org/abs/1312.4400
Going Deeper with Convolutions:https://arxiv.org/abs/1409.4842
Deep Residual Learning for Image Recognition
https://arxiv.org/abs/1512.03385
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
https://arxiv.org/abs/1506.01497
Supervised Sequence Labelling with Recurrent Neural Networks
http://www.cs.toronto.edu/~graves/preprint.pdf
Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs(Blog系列)
http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
https://arxiv.org/abs/1412.3555
A Clockwork RNN:https://arxiv.org/pdf/1402.3511.pdf
Caffe to TensorFlow(模型转换_工具)
https://github.com/ethereon/caffe-tensorflow
Deep Forest: Towards An Alternative to Deep Neural Networks
https://arxiv.org/abs/1702.08835
An implementation of neural style in TensorFlow
https://github.com/anishathalye/neural-style
Magenta is a project from the Google Brain team that asks: use machine learning to create compelling art and music.
https://github.com/tensorflow/magenta
17 Category Flower Dataset(数据集):http://www.robots.ox.ac.uk/~vgg/data/flowers/17/
Keras: The Python Deep Learning library (Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, orTheano.)
Keras: Deep Learning for Python
https://github.com/fchollet/keras
THE MNIST DATABASE of handwritten digits(数据集)
http://yann.lecun.com/exdb/mnist/
Deep Learning Face Representation from Predicting 10,000 Classes(blue)
http://mmlab.ie.cuhk.%20edu.hk/pdf/YiSun_CVPR14.pdf
Face Recognition using Tensorflow (This is a TensorFlow implementation of the face recognizer described in the paper "FaceNet: A Unified Embedding for Face Recognition and Clustering". The project also uses ideas from the paper "A Discriminative Feature Learning Approach for Deep Face Recognition" as well as the paper "Deep Face Recognition" from the Visual Geometry Group at Oxford.)
https://github.com/davidsandberg/facenet
FaceNet: A Unified Embedding for Face Recognition and Clustering
https://arxiv.org/abs/1503.03832
Labeled Faces in the Wild Home (a database of face photographs designed for studying the problem of unconstrained face recognition. The data set contains more than 13,000 images of faces collected from the web)
http://vis-www.cs.umass.edu/lfw/
Rude Carnie: Age and Gender Deep Learning with TensorFlow
https://github.com/dpressel/rude-carnie
Unfiltered faces for gender and age classification(The OUI-Adience Face Image Project)
http://www.openu.ac.il/home/hassner/Adience/data.html#agegender
Flickr: Find your inspiration(blue)
Age and Gender Classification using Convolutional Neural Networks
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.722.9654&rep=rep1&type=pdf
How-Old.net(识别性别和年龄的网站)
The Unreasonable Effectiveness of Recurrent Neural Networks
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
easy_seq2seq(make it easy for people to train their own seq2seq model with any corpus)
https://github.com/suriyadeepan/easy_seq2seq
practical_seq2seq(To make life easier for beginners looking to experiment with seq2seq model.)
https://github.com/suriyadeepan/practical_seq2seq
Cristian Danescu-Niculescu-Mizil (Cornell Movie--Dialogs Corpus)
http://www.cs.cornell.edu/~cristian/Chameleons_in_imagined_conversations.html
Grammar as a Foreign Language
https://arxiv.org/abs/1412.7449
The Syntax, Semantics and Inference Mechanism in Natural Language
http://www.aaai.org/Papers/Symposia/Fall/1996/FS-96-04/FS96-04-010.pdf
Generative Adversarial Networks
https://arxiv.org/abs/1406.2661
Conditional Image Synthesis with Auxiliary Classifier GANs
https://arxiv.org/pdf/1610.09585.pdf
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
https://arxiv.org/abs/1606.00709
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
https://arxiv.org/abs/1603.04467
tensorflow_examples (TensorFlow Example Projects)
https://github.com/tobegit3hub/tensorflow_examples
Parameter Server for Distributed Machine Learning
http://www.cs.cmu.edu/~muli/file/ps.pdf
Revisiting Distributed Synchronous SGD
https://arxiv.org/abs/1604.00981
Official docker images for deep learning framework TensorFlow (http://www.tensorflow.org)
https://hub.docker.com/r/tensorflow/tensorflow/
TensorFlowOnSpark (TensorFlowOnSpark brings scalable deep learning to Apache Hadoop and Apache Spark clusters.)
https://github.com/yahoo/TensorFlowOnSpark
数据集参考:
http://lrs.icg.tugraz.at/research/aflw/
http://vis-www.cs.umass.edu/lfw/
http://www.robots.ox.ac.uk/~vgg/data/vgg_face/
http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html
https://research.google.com/youtube8m/
https://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html
http://pascal.inrialpes.fr/data/human/
http://www.cvlibs.net/datasets/kitti/
http://www.openu.ac.il/home/hassner/Adience/data.html