Python is widely used in the field of artificial intelligence, including machine learning, deep learning, natural language processing, computer vision, etc.If you are planning to conduct practical learning through Python, you may want to know about the resources recommended in this anthology. These resources can help you systematically learn the application of Python in the field of artificial intelligence and master practical skills.
Points: 1Type: Technical documentUploader:HyperplatinumUpload time: May 30, 2021
Introduction: This book was written by Wes McKinney, the founder of Pandas project, and details the specific details and basic points of using Python to operate, process, clean and regularize data.Version 2 is a comprehensive revision and update of Python 3.6, covering the new version of pandas, NumPy, IPython and Jupyter, and adding a large number of actual cases, which can help you solve a series of data analysis problems efficiently.Major updates in Version 2 include:• All codes, including the update of Python tutorial to Python 3.6 (Python 2.7 is used in the first version)• Updated the installation guidelines for Python third-party release Anaconda and other required Python packages• Update the Pandas library to the new version in 2017• A new chapter on more advanced pandas tools and some tips• Brief introduction to the use of new statsmodes and scikit-learn
Introduction: This book skillfully combines Python language and AI knowledge for layout, so that readers can not only comprehensively learn Python programming language, but also systematically understand the basic principles of AI and deeply master AI *, which is a key technology of the new generation of AI, and is equipped with rich teaching cases and cutting-edge hot applications,Each knowledge point has a corresponding Python language implementation.The book is divided into nine chapters.Chapter 1 mainly explains the development history, driving factors and key technologies of AI.Chapter 2 is the Python programming language, which systematically introduces the syntax rules, data types, program structure, file operation and graphical programming of Python.Chapter 3 is the basis of probability and statistics, which is the theoretical basis of artificial intelligence.Chapter 4 is the * y method, which explains the essence of artificial intelligence algorithm.Chapter 5 Deep Learning and Neural Networks is the key chapter of this book. This chapter gives a comprehensive and in-depth explanation of current multi-layer neural networks based on deep learning, such as convolutional neural networks, recursive/cyclic neural networks, short-term memory neural networks, etc.Chapter 6 TensorFlow Deep Learning focuses on the use of TensorFlow, an open source framework for deep learning, and how to use TensorFlow to develop and deploy various deep learning models.Chapter 7 Data collection and data set production, details how to collect data from the Internet through web crawlers and other methods and make data sets.Chapter 8 elaborates how to use GPU parallel computing equipment and CUDA programming to accelerate the model training of AI deep learning.Chapter 9 carefully selects seven AI experiment cases, including intelligent data analysis, intelligent understanding of video images, natural language processing and other aspects. From simple to difficult, they can be used as the experimental teaching content of the book.This book can be used as a learning book for senior undergraduate and graduate students majoring in artificial intelligence, computer, electronic information, intelligent science and technology, data science and big data, robot engineering, etc., as well as a reference book for researchers, engineering technicians and intelligent application enthusiasts engaged in artificial intelligence research and development.Chapter 1 Overview1.1 Introduction1.2 Concept and definition of artificial intelligence1.3 Three schools of artificial intelligence1.3.1 Semiotic School1.3.2 Connectionist School1.3.3 Behaviorist School1.4 Origin and development of artificial intelligence1.5 Drivers of the new generation of AI1.5.1 Data volume * spontaneous growth1.5.2 Significant increase in computing capacity1.5.3 Algorithm development such as deep learning1.5.4 Mobile AI innovation application traction1.6 Key technologies of artificial intelligence1.6.1 Machine learning and deep learning1.6.2 Knowledge map1.6.3 Natural language processing1.6.4 Human computer interaction1.6.5 Computer vision1.6.6 Biometric recognition1.6.7 SLAM technology1.6.8 VR/AR/MR technologySummary of this chapterAfter class thinking questionsChapter 2 Python Programming Language2.1 Introduction to Python2.1.1 Development of Python language2.1.2 Installation of Python development environment2.1.3 Python running2.2 Basic Python syntax and data types2.2.1 Problem solving by program2.2.2 Python program syntax elements2.2.3 Common Functions2.2.4 Python Basic Data Types2.2.5 Python composite data type2.3 Python program structure2.3.1 Branch structure2.3.2 Cycle structure2.3.3 Circular reserved word2.3.4 Exception handling2.4 Python functions and modular programming2.4.1 Basic Use of Functions2.4.2 Transfer of parameters2.4.3 Return value of function2.4.4 Variable Scope2.4.5 Anonymous Functions2.4.6 Function Application2.4.7 Code reuse and modular programming2.5 Python object-oriented programming2.5.1 Definition and use of class2.5.2 Attributes and methods2.5.3 Inheritance2.6 Python file operation and graphical programming2.6.1 Basic operation of documents2.6.2 Graphical interface tkiner2.6.3 Word practice systemSummary of this chapterAfter class thinking questionsChapter 3 Basics of Probability and Statistics3.1 Probability theory3.1.1 Probability and conditional probability3.1.2 Random variable3.1.3 Discrete random variable distribution Python experiment3.1.4 Continuous random variable distribution Python experiment3.2 Basis of mathematical statistics3.2.1 Population and sample3.2.2 Statistics and sampling distribution3.2.3 Law of large numbers and central limit theorem3.3 Parameter estimation3.3.1 Point estimation3.3.2 Criteria for evaluating estimators3.3.3 Interval estimationSummary of this chapterAfter class thinking questionsChapter 4 * Optimization Methods4.1 * Optimization method basis4.1.1 * Mathematical model of optimization problem4.1.2 * Classification of optimization problems and application cases4.1.3 Mathematical Basis4.2 Convex optimization4.2.1 Convex set4.2.2 Convex function4.2.3 Convex optimization concept4.2.4 Python example4.3 * Small binary multiplication4.3.1 * Principle of small multiplication4.3.2 Python example4.4 Gradient descent method4.4.1 Gradient descent idea4.4.2 Gradient descent algorithm steps4.4.3 Classification of gradient algorithm4.4.4 Python example4.5 Newton method4.5.1 Basic principle of Newton method4.5.2 Steps of Newton method4.5.3 Newton method for solving unconstrained optimization problems4.5.4 Python example4.6 Conjugate gradient method4.6.1 Conjugate direction4.6.2 Basic principle of conjugate gradient method4.6.3 Iteration steps of conjugate gradient method4.6.4 Python ExamplesSummary of this chapterAfter class thinking questionsChapter 5 Deep Learning and Neural Networks5.1 Deep learning5.1.1 Concept of deep learning5.1.2 Principle of deep learning5.1.3 Deep learning training5.2 Basis of Artificial Neural Network5.2.1 Neuron sensor5.2.2 Neural network model5.2.3 Learning methods5.2.4 Learning rules5.2.5 Activation function5.2.6 Gradient descent method5.2.7 Cross entropy loss function5.2.8 Over fitting and under fitting5.3 Convolution neural network5.3.1 Introduction to convolutional neural network5.3.2 Convolution neural network structure5.3.3 Convolution neural network calculation5.3.4 Typical convolutional neural network5.4 Cyclic neural network5.4.1 Introduction to cyclic neural network5.4.2 Cyclic neural network structure5.4.3 Cyclic neural network calculation5.5 Long - and short-term memory network5.5.1 Introduction to short-term memory network5.5.2 Long - and short-term memory network structure5.5.3 Long - and short-term memory network calculationSummary of this chapterAfter class thinking questionsChapter 6 TensorFlow Deep Learning6.1 Introduction6.2 TensorFlow Technical Features6.3 TensorFlow Component Structure6.4 Fundamentals of TensorFlow Programming6.4.1 TensorFlow program structure6.4.2 TensorFlow programming model6.4.3 Common APIs of TensorFlow6.4.4 TensorFlow Variable Scope6.4.5 TensorFlow batch standardization6.5 TensorFlow Neural Network Model Construction6.5.1 Neuron function and optimization method6.5.2 Convolution function6.5.3 Pooling function6.5.4 Classification function6.5.5 Optimization method6.6 TensorFlow Running Environment Installation6.6.1 Python installation6.6.2 Installation of pip tools6.6.3 Sublime installation6.7 Construction of TensorFlow deep learning model6.7.1 Generating fitted data sets6.7.2 Construction of linear regression model data flow diagram6.7.3 Running the Built Data Flow Diagram in Session6.7.4 Linear regression model of output fitting6.7.5 Visualization of TensorBoard neural network data flow diagramSummary of this chapterAfter class thinking questionsChapter 7 Data Collection and Data Set Production7.1 Introduction7.2 Python data collection7.2.1 Web mechanism and crawler principle7.2.2 Python third-party library7.2.3 Three crawler libraries7.2.4 Regular expression7.2.5 Using API7.2.6 Crawler advancement7.3 Production of training data set7.3.1 Data access7.3.2 Data cleaning7.4 Data collection and data set production examplesSummary of this chapterAfter class thinking questionsChapter 8 GPU Parallel Computing and CUDA Programming8.1 Introduction8.2 GPU general calculation8.2.1 Von Neumann system architecture8.2.2 Introduction to GPU development8.2.3 Early GPGPU programming8.2.4 NVIDIA and CUDA8．3CUDA8.3.1 GPU hardware8.3.2 CPU and GPU8.3.3 Calculation capability of GPU8.3.4 CUDA software architecture8.3.5 CUDA hardware framework8.3.6 CUDA programming model8.3.7 Deep learning and GPU acceleration calculation8.3.8 CUDA environment construction under deep learning8.4 Cases of CUDA Accelerated Deep Learning8.4.1 CUDA application in TensorFlow framework8.4.2 CUDA application in PyTorch frameworkSummary of this chapterAfter class thinking questionsChapter 9 Python Artificial Intelligence Experiment9.1 Curve fitting experiment9.1.1 Test content9.1.2 Test steps9.2 Prediction of Titanic passenger death probability9.2.1 Test content9.2.2 Test steps9.3 Stock forecast9.3.1 Test content9.3.2 Test steps9.4 License plate recognition9.4.1 Test content9.4.2 Test steps9.5 Wear a mask for identification9.5.1 Test content9.5.2 Test steps9.6 Automatic poem writing experiment9.6.1 Test contents9.6.2 Test steps9.7 Chat robot experiment9.7.1 Test contents9.7.2 Test stepsSummary of this chapterAfter class thinking questions
Introduction: This book takes Python 3.5 as the programming environment, starts with the basic programming ideas, and gradually launches Python language teaching. It is a programming book for the majority of programming learners.This book uses cases to drive the explanation of knowledge points, and decomposes Python knowledge points into different cases. Each case has its own focus. At the same time, it shows the design idea and design concept of the actual project, so that readers can draw inferences from one instance.The cases in this book are practical, such as campus network search engines, small translators, and crawling Baidu pictures. With slight modifications, these crawler cases can be applied to actual projects;There are also cases of WeChat robot development through WeChat communication protocol, text classification of machine learning, handwriting recognition based on convolutional neural network, etc;In addition, there are some familiar game cases, such as Lianliankan, Pushbox, Chinese chess, online gobang, two person mahjong, figure puzzles and airplane fights.Through this book, readers will master Python programming technology and skills, learn object-oriented design methods, and understand all relevant content of program design.Chapter 1 Python Basics 11.1 Introduction to Python 11.2 Python Syntax Basics 21.2.1 Python data type 21.2.2 Sequence data structure41.2.3 Python Control Statement121.2.4 Python functions and modules181.3 Python object-oriented design 221.3.1 Definition and use class 221.3.2 Constructor231.3.3 Destructor241.3.4 Instance Properties and Class Properties241.3.5 Private Members and Public Members 251.3.6 Method 261.3.7 Inheritance of Classes 271.3.8 Polymorphism291.3.9 Object oriented application case - poker licensing program 311.4 Python graphical interface design 341.4.1 Creating Windows Windows351.4.2 Geometric Layout Manager351.4.3 Tkinter assembly391.4.4 Tkinter font 491.4.5 Python Event Handling 511.4.6 Graphic interface design application case - developing number guessing game 551.5 Use of Python Files571.5.1 Opening/Creating a Document571.5.2 Reading Text File591.5.3 Writing Text Files601.5.4 Intra file movement621.5.5 Closing of documents 631.5.6 Reading/writing binary files 641.6 Third party library of Python 66
Introduction: Python Project Development Case Collection has developed 8 development directions and 23 projects from the perspective of beginners. It allows readers to learn in practice step by step and improve their actual development ability in practice.The book is divided into 8 parts: console programs, small games, practical gadgets, web crawlers, data analysis, artificial intelligence, Web sites and WeChat/small programs, including: student information management system, enterprise code generation system, simple renju game (console version), Mary Adventures, color board airplane war, DIY character painting, drawing board toolsWord assistant, picture batch processor, RCQ reader library, train ticket analysis assistant, Gaode Map 58 rental, Mahua FunAge film and television works analysis, Excel data analyst, intelligent parking lot license plate recognition and billing system, AI intelligent contact management system, 51 MallBBS Q&A community, Sweet Orange Music Network, smart campus evaluation system, image guessing idiom applet, what to eat today applet, WeChat robot.This book will not only take you to enjoy the wonderful world of Python development, enlighten your programming thinking, but also let you enjoy the charming development charm of Python!To facilitate readers' learning, the website of Tomorrow College (www.mingrisoft.com) provides the supporting resources for this book, and the source code and related resources of the project are also provided in the cloud disk resource package. Readers can choose any way to download the resources they need;At the same time, you can also access more learning resources and technical support by logging on the website of Tomorrow College.
Introduction: A quick introduction to Python driven and grounded by local authors.This is the first practical guide to Python based on open source hardware.The development instance, complete project file and source code based on pcDuino are highly operable.Python Hardware Programming Practice is mainly aimed at Python beginners with weak computer foundation, trying to use the language style of easy to understand and easy to understand to explain the basic concepts of Python.On the premise of establishing the basic concept of Python, guide the reader to learn the choice of Python version, the construction of Python development environment under different platforms, and the basic syntax of Python step by step, and finally enable the reader to realize some simple application development with Python.Python Hardware Programming Practice is mainly applicable to beginners without Python foundation, including but not limited to engineers with hardware background, non computer professional readers, Python amateurs and students.prefaceChapter 1 Introduction to Python/11.1 What is Python/11.2 Four definitions of Python/11.2.1 A scripting language/11.2.2 An interpretive language/31.2.3 A high-level language/31.2.4 An object-oriented language/41.3 Features of Python/51.3.1 Advantages and disadvantages of being a scripting language/51.3.2 Features of Python/71.4 Python application/91.4.1 What Python can do/101.4.2 What Python is more suitable for/101.4.3 What can you do with Python/121.5 Basic Knowledge of Python/131.5.1 Suffix of Python file/131.5.2 Abbreviations and abbreviations of Python/131.5.3 Python official website/131.5.4 Python Logo/13Chapter 2 Download and Install Python/142.1 Common problems caused by inappropriate Python version/142.2 Two major Python versions/152.2.1 Python Version History/152.2.2 Difference between Python 2 and Python 3/162.3 How to select the appropriate version/202.3.1 Choose Python 2 or Python 3/212.3.2 Choose whether Python is 32-bit or 64 bit/212.4 Release format of common software/232.4.1 Source code format/232.4.2 Binary format/252.5 Download the appropriate Python installation package/262.5.1 What forms does Python provide/262.5.2 Select a more stable and faster domestic download source/292.6 How to install Python/29 in Windows2.6.1 Installing Python/29 in Windows 72.6.2 Frequently asked questions after installing Python in Windows/352.7 Installing Python/36 in Linux2.7.1 Installing Python/36 in Ubuntu2.7.2 Why not recommend beginners to install Python/37 in Ubuntu2.8 Installing Python/38 on MacChapter 3 Select the Appropriate Python Development Environment/393.1 Common things for developing Python on different platforms/393.2 Common features of Python development on different platforms/403.2.1 The most original development method of Python/413.2.2 Interactive development using Python shell/413.2.3 Developing with Python IDE/433.3 Python IDE/443.3.1 Relationship between Python IDE, editor, terminal, etc./443.3.2 Common IDE of Python/463.3.3 Frequently asked questions and answers of Python IDE/623.4 Python development in Windows/653.4.1 The most original Python development mode/653.4.2 Interactive development with Python shell/723.4.3 Developing with Python IDE/793.5 Python development in Linux/793.5.1 The most original development mode of Python/803.5.2 Interactive development with Python shell/823.5.3 Developing with Python IDE/833.6 Python development in Mac environment/833.6.1 The most original development mode of Python/833.6.2 Interactive development with Python shell/863.6.3 Developing with Python IDE/863.7 Which environment should be selected to develop Python/87Chapter 4 Basic Knowledge of Python/894.1 SheBang and Python file encoding declaration/894.1.1 #！/usr/bin/python / 894.1.2 Python file coding declaration/894.2 Indent in Python/924.2.1 Indentation of other languages only affects code aesthetics/924.2.2 Python indentation will affect code logic/934.3 Meaning of __name__ and __main__ in Python/984.3.1 __name__ Details/984.3.2 __main__ Details/994.3.3 Purpose of using __name__ and __main__ together/994.4 Object oriented programming in Python/1034.4.1 Meaning of self and __init__/1034.4.2 Beginners should not pay too much attention to object-oriented/1094.5 Variables in Python/1094.5.1 Declaration and definition of basic variables/1094.5.2 Variable Scope/1124.6 Branch Structure in Python/1154.7 Functions in Python/116Chapter 5 Some Interesting Python Experiments/1185.1 View system platform information in Python/1185.2 Python Processing Harmonic and Signal Transformation/1195.3 More useful and interesting Python syntax/1235.3.1 Exchange different variable values in Python/1245.3.2 Slicing of variables of set class in Python/1245.3.3 For Loop and Enumerator in Python/1255.3.4 Conditional assignment in Python/126Chapter 6 Common Python Application Examples/1276.1 Python Application in Network/1276.2 Application of Python in Graphic Interface/1326.2.1 Common GUI graphic library of Python/1326.2.2 Python GUI graphics library: PyQt/1326.3 Python Application in Database/136Chapter 7 Python and Open Source Hardware/1417.1 Relationship between Python and open source hardware/1417.2 Basic Knowledge of pcDuino/1417.2.1 What is open source hardware/1417.2.2 Common open source hardware/1427.2.3 Why pcDuino/1467.2.4 How to configure open source hardware pcDuino/1477.3 Using Python/155 on open source hardware pcDuino7.3.1 Web server/1567.3.2 Water leakage monitoring/1627.3.3 Using Z-Wave to Realize Smart Home/166Appendix A How to Use Python Related Resources/174Appendix B How to continue to learn Python/181 in depthAppendix C Python Learning Materials/182
Points: 1Type: Technical documentUploader:sigmaUpload time: March 12, 2022
Introduction: Python is the preferred language for data analysis, and there is a lot of data and information in the network. How can I get the data and information I need from it?The simplest and direct way is to use crawler technology to solve the problem.Learning Python Crawler from scratch is a primer for beginners to learn how to crawl network data and information.The book contains not only Python related content, but also data processing and data mining.The content of this book is very practical. The explanation is interspersed with 22 practical cases of reptiles, which can greatly improve the practical ability of readers.This book is divided into 12 chapters. The core topics include the introduction to Python zero basic syntax, crawler principles and web page construction, my first crawler program, regular expressions, Lxml library and Xpath syntax, using API, database storage, multi process crawler, asynchronous loading, form interaction and simulated login, Selenium simulation browser, and Scrapy crawler framework.In addition, through some typical crawler cases, the book explains the method of making maps and charts with longitude and latitude information and word clouds, so that readers can experience the fun behind the data.This book is suitable for beginners, enthusiasts and related students in colleges and universities of reptile technology, as well as data crawler engineers as reference books, and also suitable for various Python data analysis training institutions as teaching materialsChapter 1 Introduction to Python Zero Basic Syntax 1Chapter 2 Principles of Crawler and Web Page Construction 17Chapter 3 My First Crawler 26Chapter 4 Regular Expression45Chapter 5 Lxml Library and Xpath Syntax 63Chapter 6 Using API 88Chapter 7 Database Storage 109Chapter 8 Multi process Crawler 139Chapter 9 Asynchronous Loading 159Chapter 10 Form Interaction and Simulated Login 182Chapter 11 Selenium Simulation Browser209Chapter 12 Scrapy Crawler Framework229
Points: 1Type: Application DocumentUploader:HyperplatinumUpload time: May 30, 2021
Introduction: This book uses simple and powerful Python language to introduce network data collection, and provides comprehensive guidance for collecting various data types in the new network.The first part focuses on the basic principles of network data collection: how to request information from the network server with Python, how to basically process the server's response, and how to interact with the website by means of automation.The second part introduces how to use web crawlers to test websites, automate processing, and how to access the network in more ways.
Points: 1Type: Technical documentUploader:sigmaUpload time: August 30, 2020
Introduction: Python Machine Learning and Practice: The Road to Kaggle Competition from scratch, written by Fan Miao/Li Chao, published in 2016.Python machine learning and practice is for all readers who are interested in the practice and competition of machine learning and data mining. From scratch, based on Python programming language, and without involving a large number of mathematical models and complex programming knowledge, we will gradually lead readers to become familiar with and master the popular machine learning, data mining and natural language processing tools, such as Scikit learningNLTK, Pandas, gensim, XGBoost, Google Tensorflow, etc.The book is divided into four chapters.Chapter 1 Introduction, introducing machine learning concepts and Python programming knowledge;Chapter 2, Basic, describes how to use Scikit learn as a basic machine learning tool;Chapter 3, Advanced Chapter, deals with how to further improve the performance of existing machine learning systems with advanced technologies or models;Chapter 4 Contest, taking the Kaggle platform as the object, helps readers use the models and techniques introduced in this book step by step to complete three representative competition tasks.Python machine learning and practice directoryChapter 1 IntroductionChapter 2 BasicChapter 3 Advanced Level ChapterChapter 4 Actual Combat
Introduction: The topic of Python and machine learning is so broad that it is impossible to cover all aspects with only one book, even if a series is produced.As far as machine learning is concerned, its fields include but are not limited to: supervised learning, unsupervised learning and semi supervised learning.The specific problems can be roughly divided into two categories: classification and regression.Python itself has many third-party libraries for machine learning, but Python and Machine Learning: Decision Tree, Integrated Learning, Support Vector Machine and Neural Network Algorithm Details and Programming Implementation only uses Numpy, a scientific computing library based on Numpy, to implement algorithm code in most cases.The purpose of this is to hope that readers can better understand the details of machine learning algorithms and understand various applications of Numpy from the implementation process.However, as a supplement, Python and Machine Learning: Decision Tree, Integrated Learning, Support Vector Machine and Neural Network Algorithm Details and Programming Implementation will apply the model in the mature third-party library scikit-lear when appropriate.Python and Machine Learning Practice: Decision Tree, Integrated Learning, Support Vector Machine and Neural Network Algorithm Details and Programming Implementation is applicable to students and practitioners who want to know about traditional machine learning algorithms, programmers who want to know how to efficiently implement machine algorithms, and employees and managers who want to know how machine learning algorithms can be applied.catalogChapter 1 Introduction to Python and Machine Learning 11.1 Introduction to machine learning 11.1.1 What is machine learning 21.1.2 Common terms of machine learning 31.1.3 Importance of machine learning 61.2 Life is short, I use Python 71.2.1 Why Python 71.2.2 Advantages of Python in machine learning 81.2.3 Installation and use of Anaconda 81.3 The first machine learning example 121.3.1 Obtaining and processing data 131.3.2 Selection and training model 141.3.3 Evaluation and visualization results 151.4 Summary of this chapter 17Chapter 2 Bayesian Classifier 182.1 Bayesian School 182.1.1 Bayesian School and Frequency School 192.1.2 Bayesian Decision Theory 192.2 Parameter estimation 202.2.1 Maximum likelihood estimation (ML estimation) 212.2.2 Maximum a posteriori probability estimation (MAP estimation) 222.3 Naive Bayes 232.3.1 Algorithm statement and basic architecture building 232.3.2 Implementation and evaluation of MultinomialNB 312.3.3 Implementation and evaluation of GaussianNB 402.3.4 Implementation and evaluation of MergedNB 432.3.5 Vectorization of algorithm502.4 Semi naive Bayesian and Bayesian network532.4.1 Semi naive Bayesian 532.4.2 Bayesian network542.5 Relevant mathematical theories552.5.1 Bayesian formula and posterior probability 552.5.2 Discrete Naive Bayesian Algorithm562.5.3 Naive Bayes and Bayesian Decision 582.6 Summary of this chapter 59Chapter 3 Decision Tree 603.1 Information of data 603.1.1 Introduction to information theory613.1.2 Uncertainty 613.1.3 Information gain 653.1.4 Generation of Decision Tree 683.1.5 Related Realization 773.2 Over fitting and pruning 923.2.1 Pruning algorithm for ID3 and C4.5 933.2.2 CART pruning 1003.3 Evaluation and visualization 1033.4 Relevant mathematical theories1113.5 Summary of this chapter 113Chapter 4 Integrated Learning 1144.1 The idea of "integration" 1144.1.1 It's easy for all to lift 1154.1.2 Bagging and random forest 1154.1.3 PAC Framework and Boosting 1194.2 Random forest algorithm 1204.3 AdaBoost algorithm1244.3.1 AdaBoost algorithm statement 1244.3.2 Selection of weak model1264.3.3 Implementation of AdaBoost1274.4 Performance analysis of integrated model1294.4.1 Performance on Random Datasets 1304.4.2 Performance on XOR Dataset1314.4.3 Performance on Spiral Datasets 1344.4.4 Performance on Mushroom Datasets 1364.5 Interpretation of AdaBoost algorithm1384.6 Relevant mathematical theories1394.6.1 Empirical distribution function 1394.6.2 AdaBoost and forward step-by-step addition model1404.7 Summary of this chapter 142Chapter 5 Support Vector Machine1445.1 Perceptron model1455.1.1 Linear separability and perceptron strategy 1455.1.2 Perceptron algorithm1485.1.3 Dual form of perceptron algorithm1515.2 From perceptron to support vector machine1535.2.1 Interval maximization and linear SVM 1545.2.2 Dual form of SVM algorithm 1585.2.3 SVM training1615.3 From linear to nonlinear 1635.3.1 Brief Introduction to Nuclear Skill 1635.3.2 Application of nuclear skills 1665.4 Multi classification and support vector regression 1805.4.1 One vs Test 1805.4.2 One to one method (One vs One) 1815.4.3 Directed Acyclic Graph Method1815.4.4 Support Vector Regression 1825.5 Relevant mathematical theories1835.5.1 Gradient descent method1835.5.2 Lagrange duality 1855.6 Summary of this chapter 187Chapter 6 Neural Network 1886.1 From perceptron to multi-layer perceptron 1896.2 Forward conduction algorithm1926.2.1 Algorithm overview 1936.2.2 Activation Function1956.2.3 Cost Function 1996.3 Back propagation algorithm2006.3.1 Algorithm overview 2006.3.2 Selection of loss function2026.3.3 Relevant realization2056.4 Special layer structure 2116.5 Updating of parameters2146．5．1 Vanilla Update 2176．5．2 Momentum Update 2176．5．3 Nesterov Momentum Update 2196．5．4 RMSProp 2206．5．5 Adam 2216．5．6 Factory 2226.6 Simple network structure2236.7 Network Structure under "Big Data" 2276.7.1 The idea of batch2286.7.2 Cross validation 2306.7.3 Progress bar2316.7.4 Timer 2336.8 Relevant mathematical theories2356.8.1 Deduction of BP algorithm2356.8.2 Softmax+log Likelihood Combination2386.9 Summary of this chapter 240Chapter 7 Convolutional Neural Network2417.1 From NN to CNN 2427.1.1 Sharing of "vision" 2427.1.2 Forward conduction algorithm2437.1.3 Fully Connected Layer2507.1.4 Pooling 2517.2 Rewrite NN 252 with TensorFlow7.2.1 Back propagation algorithm2527.2.2 Rewrite Layer structure2537.2.3 Implementing SubLayer structure2557.2.4 Rewrite CostLayer structure2617.2.5 Rewrite network structure2627.3 Expand NN to CNN 2637.3.1 Realization of convolution layer2637.3.2 Realization of pooling layer2667.3.3 Realization of special layer structure in CNN 2677.3.4 Implementing LayerFactory 2687.3.5 Extended network structure2707.4 CNN performance2727.4.1 Problem description2727.4.2 Building CNN model2737.4.3 Model analysis 2807.4.4 Method of applying CNN 2837．4．5 Inception 2867.5 Summary of this chapter 289Appendix A Introduction to Python 290Appendix B Introduction to Numpy 303Appendix C Introduction to TensorFlow 310
Introduction: This book mainly introduces how to use matplotlib, Seaborn, plotnine, Basemap and other packages in Python to draw professional charts.This book first introduces the basic knowledge of Python language programming, as well as NumPy and Pandas' data operation methods;Then compare and introduce the graphic syntax of matplotlib, Seaborn and plotnine.This book systematically introduces the methods of using matplotlib, Seaborn and plotnine to draw common two-dimensional and three-dimensional charts such as category comparison, data relationship, time series, overall and local, and geospatial charts.In addition, this book also introduces the specifications and differences between business charts and academic charts, and how to use matplotlib to draw HTML interactive page animations.catalogChapter 1 Python Programming Basics 11.1 Basic knowledge of Pythontwo1.1.1 Installation of Python 3.7two1.1.2 Package installation and usethree1.1.3 Basic Python Operationsfour1.2 Six common data structures 51.2.1 Listfive1.2.2 Dictionarysix1.2.3 Tuplesix1.3 Control statement and function writingsix1.3.1 Control statementsix1.3.2 Function WritingeightChapter 2 Data Processing Basisten2.1 NumPy: numerical operation 112.1.1 Creating Arrays 112.1.2 Index and transformation of arraytwelve2.1.3 Combination of arraysthirteen2.1.4 Statistical Functions of Arraysfourteen2.2 Pandas: Table processing 152.2.1 Series data structurefifteen2.2.2 Data structure: DataFrame 162.2.3 Data type: Categoricaleighteen2.2.4 Table transformationnineteen2.2.5 Transformation of variablestwenty2.2.6 Sorting of tablestwenty2.2.7 Table splicingtwenty-one2.2.8 Fusion of tablestwenty-two2.2.9 Table grouping operationtwenty-three2.2.10 Data import and exporttwenty-six2.2.11 Treatment of missing valuestwenty-eightChapter 3 Fundamentals of Data Visualization 293.1 matplotlib . thirty-three3.1.1 Graphic objects and elementsthirty-three3.1.2 Common chart typesthirty-six3.1.3 Drawing of sub graphthirty-eight3.1.4 Coordinate system transformationforty-one3.1.5 Chart exportforty-four3.2 Seaborn 443.2.1 Common chart typesforty-five3.2.2 Chart style and color themeforty-six3.2.3 Sectional drawing of chartforty-eight3.3 plotnine 503.3.1 geom_???() and stat_???() 513.3.2 Mapping of aesthetic parametersfifty-four3.3.3 Measurement Adjustmentfifty-eight3.3.4 Coordinate system and its measurementsixty-four3.3.5 Legendsixty-nine3.3.6 Theme systemseventy-one3.3.7 Parting systemseventy-three3.3.8 Position adjustmentseventy-four3.4 Application principle of visual colorseventy-six3.4.1 RGB color mode763.4.2 HSL color modeseventy-seven3.4.3 LUV color mode793.4.4 Matching principle of color themeeighty3.4.5 Pick up and use of color theme schemeeighty-four3.4.6 Application case of color themeeighty-seven3.5 Basic types of chartsninety-one3.5.1 Category comparisonninety-one3.5.2 Data relationshipninety-two3.5.3 Data distributionninety-three3.5.4 Time seriesninety-four3.5.5 Local and overallninety-four3.5.6 Geospaceninety-fiveChapter 4 Category Comparison Chart 964.1 Column chart seriesninety-seven4.1.1 Single data series bar chartninety-eight4.1.2 Column chart of multiple data seriesone hundred4.1.3 Stacked Column Chartone hundred and one4.1.4 Percentage Stacked Column Chartone hundred and two4.2 Bar chart seriesone hundred and four4.3 Unequal width column chartone hundred and five4.4 Cleveland Point Mapone hundred and six4.5 Slope mapone hundred and eight4.6 Nightingale rose diagram 1104.7 Radial column diagram1144.8 Radar draw1174.9 Word Cloud Picture 119Chapter 5 Data Relational Chart1225.1 Scatter chart seriesone hundred and twenty-three5.1.1 Two dimensional scatter chart of trend displayone hundred and twenty-three5.1.2 Two dimensional scatter diagram of distribution displayone hundred and thirty-one5.1.3 Bubble Chartone hundred and thirty-six5.1.4 Three dimensional scatter diagramone hundred and thirty-nine5.2 Surface fittingone hundred and forty-two5.3 Contour mapone hundred and forty-five5.4 Scatter plot seriesone hundred and forty-seven5.5 Waterfall Diagramone hundred and forty-nine5.6 Correlation coefficient diagramone hundred and fifty-sixChapter 6 Data Distribution Chart 1596.1 Statistical Histogram and Kernel Density Estimation Chartone hundred and sixty-one6.1.1 Statistical histogramone hundred and sixty-one6.1.2 Kernel density estimation diagramone hundred and sixty-one6.2 Data Distribution Chart Seriesone hundred and sixty-five6.2.1 Scatter data distribution map seriesone hundred and sixty-six6.2.2 Column distribution diagram seriesone hundred and sixty-eight6.2.3 Box diagram seriesone hundred and sixty-nine6.2.4 Violin Diagramone hundred and seventy-five6.3 Two dimensional statistical histogram and kernel density estimation chartone hundred and seventy-nine6.3.1 Two dimensional statistical histogramone hundred and seventy-nine6.3.2 Two dimensional kernel density estimation diagramone hundred and eightyChapter 7 Time Series Chart 1847.1 Line chart and area chart seriesone hundred and eighty-five7.1.1 Line chartone hundred and eighty-five7.1.2 Area chartone hundred and eighty-five7.2 Calendar Chartone hundred and ninety-two7.3 Quantified oscillogramone hundred and ninety-fiveChapter 8 Local Integral Chart1998.1 Pie chart seriestwo hundred8.1.1 Pie charttwo hundred8.1.2 Ring charttwo hundred and two8.2 Mosaic diagramtwo hundred and three8.3 Waffle pie charttwo hundred and six8.4 Block/Point Column Chart Series 208Chapter 9 High dimensional Data Chart2139.1 Transformation display of high-dimensional datatwo hundred and fifteen9.1.1 Principal component analysistwo hundred and fifteen9.1.2 t-SNE algorithm2179.2 Sectional drawingtwo hundred and eighteen9.3 Matrix Scatter Charttwo hundred and twenty-one9.4 Thermal diagramtwo hundred and twenty-four9.5 Parallel coordinate system diagramtwo hundred and twenty-seven9.6 RadViz Figure 229Chapter 10 Geospatial Chart23110.1 Maps of different levelstwo hundred and thirty-two10.1.1 World maptwo hundred and thirty-two10.1.2 Country Maptwo hundred and thirty-eight10.2 Grading statistical maptwo hundred and forty-one10.3 Point tracing maptwo hundred and forty-four10.4 Maps with columnstwo hundred and forty-eight10.5 Isometric maptwo hundred and fifty10.6 Point maptwo hundred and fifty-two10.7 Simplified schematic diagramtwo hundred and fifty-six10.8 Post mark methodtwo hundred and sixtyChapter 11 Data Visualization Cases 26311.1 Example of business chartingtwo hundred and sixty-four11.1.1 Basis of business charting26411.1.2 Cases of business charting ① 26911.1.3 Business diagram drawing cases ② 27011.2 Example of academic chart drawingtwo hundred and seventy-three11.2.1 Fundamentals of academic charting27411.2.2 Cases of academic charting27611.3 Data analysis and visualization casestwo hundred and seventy-eight11.3.1 Drawing of schematic subway route map 27811.3.2 Drawing of actual subway line map28011.3.3 Application of subway route map28111.4 Visual demonstration of dynamic datatwo hundred and eighty-six11.4.1 Making of dynamic bar chart28611.4.2 Preparation of dynamic area map29111.4.3 Production of 3D cylindrical map animation296reference.three hundred and one
Introduction: This book is intended for programmers who have programming knowledge of other high-level programming languages. It introduces today's very attractive and advanced computing technology and Python programming through practical examples.By learning more than 500 practical examples provided in this book, readers will learn to use the interactive IPython interpreter and Jupyter Notebook and quickly master Python coding methods.After learning the basic knowledge of Python in Chapters 1-5 and some key contents in Chapters 6 and 7, the reader will be able to process the important practical contents of AI cases in Chapters 11-16, including natural language processing, Twitter data mining for emotion analysis, cognitive computing using IBM Watson, supervised machine learning using classification and regression,Unsupervised machine learning through clustering, computer vision based on deep learning and convolutional neural network, deep learning based on recurrent neural network, big data processing based on Hadoop, Spark and NoSQL databases, Internet of Things, etc.Readers will also directly or indirectly use cloud based services, such as Twitter, Google Translate, IBM Watson, Microsoft Azure, OpenMapQuest, PubNub, etc.Reviewer commentsTranslator's PrefaceprefaceAbout the authorBefore starting to read this bookPart I Basic Python Knowledge Quick StartChapter 1 Python and Big Data Overview 21.1 Introduction 21.2 Quickly review the basic knowledge of object-oriented technology31.3 Python51.4 Python Library 71.4.1 Python standard library 71.4.2 Data science database 81.5 Try IPython and Jupyter Notebook91.5.1 Using the IPython interaction mode as a calculator 101.5.2 Executing Python programs with the IPython interpreter 111.5.3 Writing and executing code in Jupyter Notebook 121.6 Cloud and Internet of Things 161.6.1 Cloud 161.6.2 Internet of Things 171.7 How big the big data is 181.7.1 Big Data Analysis 221.7.2 Data science and big data are bringing changes: use case 231.8 Case study: big data mobile applications241.9 Introduction to Data Science: Artificial Intelligence - an interdisciplinary between computer science and data science 261.10 Summary 28Chapter 2 Python Program Design Overview 292.1 Introduction 292.2 Variables and assignment statements302.3 Arithmetic operation312.4 print functions, single quotation marks and double quotation marks342.5 Triple quoted string 362.6 Obtaining Inputs from the User372.7 Decision: if statement and comparison operator392.8 Objects and dynamic types432.9 Introduction to Data Science: Basic Descriptive Statistics 442.10 Summary 46Chapter 3 Control Statement483.1 Introduction 483.2 Overview of Control Statement493.3 if statement493.4 if...Else and ifElif... else statement 503.5 While Statement533.6 for statement543.6.1 Iteratible Objects, Lists, and Iterator 553.6.2 Built in function range553.7 Enhanced Assignment563.8 Sequence Control Iteration and Formatting String563.9 Iteration of boundary value control573.10 Built in function range: in-depth discussion 593.11 Using Decimal Type to Process Currency Amounts 593.12 Break and continue statement633.13 Boolean operators and, or and not633.14 Introduction to Data Science: Centralized Trend Measurement - Mean, Median and Mode 663.15 Summary 67Chapter 4 Functions 694.1 Introduction 694.2 Function definition704.3 Multi parameter function724.4 Generation of random numbers744.5 Case study: a luck game 764.6 Python standard library 794.7 Functions in math module804.8 Using tab auto completion in IPython 814.9 Default parameter value824.10 Keyword parameter834.11 Variable length parameter list834.12 Method: Functions belonging to object844.13 Scope rule854.14 import: in-depth discussion 874.15 Transfer parameters to function: in-depth discussion884.16 Recursion 914.17 Functional programming934.18 Introduction to Data Science: Off center Trend Measurement 954.19 Summary 96Chapter 5 Sequence: Lists and tuples 975.1 Introduction 975.2 List985.3 Tuples1025.4 Sequence unpacking1045.5 Sequence slicing 1065.6 Using del declaration1085.7 Passing lists to functions 1095.8 List sorting 1105.9 Sequence search 1115.10 Other methods of listing 1135.11 Using List Simulation Stack1155.12 List deduction1165.13 Generator expression1185.14 Filtering, mapping and reduction 1185.15 Other sequence processing functions1205.16 Two dimensional list1225.17 Introduction to Data Science: Simulation and Static Visualization 1245.17.1 Legend of 600, 60000, 600000 dice roll 1245.17.2 Realize the visualization of the number and percentage of different points in dice1265.18 Summary 132Part II Python Data Structure, String and FileChapter 6 Dictionary and Collection1366.1 Introduction 1366.2 Dictionary1376.2.1 Creating a Dictionary1376.2.2 Traversal dictionary 1386.2.3 Basic dictionary operations1386.2.4 Keys and values method of dictionary 1406.2.5 Comparison of dictionaries 1416.2.6 Example: student achievement dictionary 1426.2.7 Example: word count 1436.2.8 Dictionary update method1446.2.9 Dictionary derivation 1456.3 Collection1466.3.1 Comparison of Collections 1476.3.2 Mathematical operation of set1486.3.3 Variable operators and methods of set1506.3.4 Collective derivation 1516.4 Introduction to Data Science: Dynamic Visualization1516.4.1 Working principle of dynamic visualization1526.4.2 Realize dynamic visualization1546.5 Summary 156Chapter 7 Array Oriented Programming with NumPy 1587.1 Introduction 1587.2 Creating arrays from existing data1597.3 Array properties1607.4 Filling an array with a specific value1627.5 Creating Arrays from Range1627.6 Performance comparison between list and array: introduce% timeit1647.7 Array operator1657.8 NumPy calculation method1677.9 General function1687.10 Index and section1707.11 View: light copy 1717.12 View: deep copy 1737.13 Remodeling and transposition 1747.14 Introduction to data science: pandas Series and DataFrame1767.14.1 Series1777.14.2 DataFrame1817.15 Summary 188Chapter 8 String: In depth discussion 1908.1 Introduction 1908.2 Formatting String1918.2.1 Representation type1918.2.2 Field width and alignment 1938.2.3 Number Formatting 1938.2.4 Format method of string 1948.3 Splicing and Repeating Strings1958.4 Removing white space characters from a string 1968.5 Character case convertion1968.6 String comparison operator1978.7 Finding substrings 1978.8 Replacing substrings1998.9 String splitting and concatenation1998.10 String test method2018.11 Original String2028.12 Introduction to Regular Expression2028.12.1 Re module and fullmatch function2038.12.2 Replacing substrings and splitting strings2078.12.3 Other search functions, access matching 2078.13 Introduction to Data Science: pandas, regular expressions and data governance 2108.14 Summary 214Chapter 9 Documents and Exceptions2159.1 Introduction 2159.2 Documents2169.3 Text file process2179.3.1 Writing Data to a Text File: Introduction to the With Statement2179.3.2 Reading data from text file2189.4 Updating text files2209.5 Serializing with JSON2219.6 Focus on security: pickle serialization and deserialization2249.7 Additional notes on document2249.8 Handling Exceptions2259.8.1 Division by zero and invalid input2269.8.2 Try statement2269.8.3 Catching multiple exceptions in one exception clause 2299.8.4 What exception is thrown by a function or method2299.8.5 What code should be written in the statement sequence of the try clause 2299.9 finally clause2299.10 Explicitly throw an exception 2319.11 (Optional) Stack expansion and backtracking 2329.12 Getting Started with Data Science: Using CSV Files2349.12.1 Python standard library module csv2349.12.2 Reading CSV file data into pandas DataFrame 2369.12.3 Reading Titanic disaster data set2379.12.4 Simple data analysis using Titanic disaster dataset 2389.12.5 Histogram of passenger age2399.13 Summary 240Part III Python Advanced TopicsChapter 10 Object Oriented Programming24210.1 Introduction 24210.2 Custom Account Class24410.2.1 Trial Account Class24510.2.2 Definition of Account class24610.2.3 Composition: Object Reference as a Class Member 24810.3 Attribute access control24810.4 property249 for data access10.4.1 Trial time class 24910.4.2 Definition of Time class25110.4.3 Design description of Time class definition25410.5 Simulate "Private" Properties25510.6 Case study: shuffle and split simulation 25710.6.1 Trial Cards and DeckOfCards 25710.6.2 Card class: import class attributes25810.6.3 DeckOfCards class 26010.6.4 Display poker images using Matplotlib 26210.7 Inheritance: base classes and subclasses 26510.8 Building inheritance hierarchy: introducing polymorphism 26710.8.1 Base Class CommissionEmployee26710.8.2 Subclass SalariedCommission Employee27010.8.3 Dealing with Commission Employee and SalariedComm-issionEmployee27310.8.4 Description of object-oriented and object-oriented programming27410.9 Duck type and polymorphism 27410.10 Operator overload27610.10.1 Trial use of Complex class27710.10.2 Definition of Complex class27810.11 Exception Class Hierarchy and Custom Exceptions27910.12 Named tuples 28010.13 Introduction to the new data class of Python 3.7 28110.13.1 Creating Card data class28210.13.2 Using Card data class28410.13.3 Advantages of data class over named tuples 28610.13.4 Advantages of data type over traditional type 28610.14 Unit testing using document strings and doctest 28610.15 Namespace and scope29010.16 Introduction to Data Science: Time Series and Simple Linear Regression 29310.17 Summary 300Part IV Case Study of Artificial Intelligence, Cloud and Big DataChapter 11 Natural Language Processing 30411.1 Introduction 30411.2 TextBlob30511.2.1 Creating a TextBlob object30711.2.2 Marking text as sentences and words30711.2.3 Part of Speech Tag30811.2.4 Extracting noun phrases 30911.2.5 Using TextBlob's default emotion analyzer for emotion analysis30911.2.6 Using NaiveBayesAnalyzer for Emotion Analysis31011.2.7 Language detection and translation31111.2.8 Deformation: complex and singular 31211.2.9 Spelling and spelling correction31311.2.10 Normalization: stem extraction and word form restore31411.2.11 Word frequency31411.2.12 Obtaining word definitions, synonyms and antonyms from WordNet 31511.2.13 Deleting stop word31711.2.14 n yuan 31811.3 Visualize word frequency using histogram and word cloud 31911.3.1 Visualize word frequency with pandas 31911.3.2 Visualize word frequency with word cloud 32111.4 Readability evaluation using Textatic library 32411.5 Using SpaCy Named Entity Recognition 32611.6 Using spaCy for similarity detection32711.7 Other NLP libraries and tools32811.8 Machine learning and deep learning natural language application32811.9 Natural Language Dataset32911.10 Summary 329Chapter 12 Twitter Data Mining 33112.1 Introduction 33112.2 Overview of Twitter API 33312.3 Creating a Twitter account 33412.4 Obtaining Twitter credentials and creating an application33412.5 What is Tweet33612.6 Tweepy33912.7 Twitter authentication via Tweepy34012.8 Getting information about a Twitter account34112.9 About Tweepy Cursor: Get followers and friends of an account 34312.9.1 Identifying followers of an account34312.9.2 Determining the target of an account 34512.9.3 Get the latest tweets of a user34512.10 Search for the latest tweet34612.11 Hot topic discovery: Twitter hot topic API34812.11.1 Places with hot topics 34812.11.2 Getting a list of hot topics 34912.11.3 Create word cloud based on popular topics 35112.12 Cleaning or pretreatment before tweet analysis35212.13 Twitter Streaming API35312.13.1 Creating a subclass of StreamListener35312.13.2 Start stream process35612.14 Tweet sentiment analysis35712.15 Geocoding and mapping 36112.15.1 Obtaining and mapping tweets36212.15.2 Utility functions in tweetutilies.py 36612.15.3 LocationListener class36712.16 How to store tweets 36812.17 Twitter and Time Series 36912.18 Summary 369Chapter 13 IBM Watson and Cognitive Computing 37013.1 Introduction 37013.2 IBM Cloud Account and Cloud Console37213.3 Watson Service37213.4 Additional services and tools37513.5 Watson Developer Cloud Python SDK37713.6 Case Study: Traveler Translation Companion APP37713.6.1 Preparation37813.6.2 Running APP37913.6.3 SimpleLanguageTranslator.py script code analysis 38013.7 Watson resource39013.8 Summary 391Chapter 14 Machine Learning: Classification, Regression and Clustering 39214.1INTRODUCTION39214.1.1 scikit-learn39314.1.2 Categories of machine learning 39414.1.3 Built in data set in scikit learn39614.1.4 Steps of typical data science research39614.2 Case study: classification using k-nearest neighbor algorithm and Digits dataset (Part 1) 39714.2.1 k-nearest neighbor algorithm39814.2.2 Loading a Dataset39914.2.3 Visual data40214.2.4 Split data for training and test40414.2.5 Creating model40514.2.6 Training model40514.2.7 Forecast number category40614.3 Case study: classification using k-nearest neighbor algorithm and Digits dataset (Part 2) 40714.3.1 Model accuracy indicator40714.3.2 k-fold cross validation 41014.3.3 Running multiple models to find the best model41114.3.4 Super parameter adjustment41314.4 Case study: time series and simple linear regression 41314.5 Case Study: Multiple Linear Regression Based on California House Price Dataset 41814.5.1 Loading a Dataset41814.5.2 Exploring Data with Pandas 42014.5.3 Visual feature42214.5.4 Split data for training and test42614.5.5 Training model42614.5.6 Test model42714.5.7 Visual prediction of house price and expected house price 42714.5.8 Regression model index42814.5.9 Selecting the best model42914.6 Case study: unsupervised learning (Part 1) - dimensionality reduction 43014.7 Case study: unsupervised learning (Part 2) - k-means clustering 43314.7.1 Loading Iris Dataset43514.7.2 Exploring Iris data set: descriptive statistics using pandas 43614.7.3 Using Seaborn's pairplot visualization dataset 43814.7.4 Using KMeans Estimator 44014.7.5 Principal component analysis dimensionality reduction 44214.7.6 Selecting the best cluster estimator 44414.8 Summary 445Chapter 15 Deep Learning 44715.1 Introduction 44715.1.1 In depth learning application44915.1.2 Deep learning demonstration 45015.1.3 Keras resources45015.2 Keras Built in Dataset45015.3 Customizing Anaconda environment45115.4 Neural network45215.5 Tensor45415.6 Convolutional neural network for vision: multi classification using MNIST dataset 45515.6.1 Loading MNIST Dataset45715.6.2 Data Exploration45715.6.3 Data preparation45915.6.4 Creating Neural Network Model46115.6.5 Training and evaluation model46815.6.6 Saving and Loading Model47215.7 Visualize the training process of neural network with TensorBoard 47315.8 ConvnetJS: browser based deep learning training and visualization 47615.9 Recurrent neural networks for sequences: sentiment analysis using IMDb datasets 47715.9.1 Loading IMDb Film Review Dataset47815.9.2 Data Exploration47815.9.3 Data preparation48015.9.4 Creating Neural Network48115.9.5 Training and evaluation model48315.10 Adjusting the deep learning model48415.11 CNN model pre trained on ImageNet 48515.12 Summary 486Chapter 16 Big Data: Hadoop, Spark, NoSQL and IoT48816.1 Introduction 48816.2 Relational database and structured query language49216.2.1 Books database49316.2.2 SELECT query49716.2.3 WHERE clause49716.2.4 ORDER BY clause49816.2.5 Consolidate data from multiple tables: INNER JOIN49916.2.6 INSERT INTO statement50016.2.7 UPDATE statement50116.2.8 DELETE FROM statement50216.3 Overview of NoSQL and NewSQL big data database50216.3.1 NoSQL Key Value Database 50316.3.2 NoSQL document database50316.3.3 NoSQL columnar database50416.3.4 NoSQL diagram database50416.3.5 NewSQL database50516.4 Case study: MongoDB JSON document database 50616.4.1 Creating MongoDB Atlas Cluster50616.4.2 Saving Tweets to MongoDB50716.5 Hadoop51516.5.1 Overview51616.5.2 Summarize the word length in Romeo-AndJuliet.txt through MapReduce 51816.5.3 Create Apache Hadoop cluster in Windows Azure HDInsight 51816.5.4 Hadoop Stream52016.5.5 Implementing mapper52016.5.6 Implementing reductor52116.5.7 Preparing to run MapReduce example52216.5.8 Running MapReduce job52316.6 Spark52516.6.1 Overview52516.6.2 Docker and Jupyter Docker Stack52616.6.3 Word count using Spark52916.6.4 Spark word count on Windows Azure 53216.7 Spark stream: use the pyspark notebookDocker stack to calculate Twitter topic tags 53516.7.1 Streaming Tweets to Socket53516.7.2 Summarize the topic tag of tweets and introduce Spark SQL53816.8 Internet of Things and Dashboard54316.8.1 Publishing and subscribing 54516.8.2 Visualizing PubNub Sample Real Time Streams with Freeboard Dashboard 54516.8.3 Simulate a thermostat connected to the Internet with Python 54716.8.4 Creating a dashboard using freeboard.io 54916.8.5 Creating a Python PubNub Subscriber 55016.9 Summary 554Index 556
Introduction: In the era of big data, big data analysis is a key technology.Python is an excellent big data analysis software. This book combines Python 3 with third-party open source tools to analyze big data, and programs to achieve data extraction, processing, analysis and visualization at the minimum cost.This book is divided into eight chapters. First, it introduces the background and industry application of big data analysis, and gives the analysis of data feature algorithm;Then, based on Python 3, introduce the scenario applications of typical third-party big data analysis tools;Finally, the big data analysis algorithm and classic example application are described in detail.This book is suitable for researchers engaged in big data analysis, practitioners of computer or mathematics and other related majors for reference study, and can also be used as a professional book for senior undergraduate or graduate students of computer or mathematics and other majors.Chapter 1 Overview of Big Data Analysis 11.1 Background of big data analysis 11.2 Application of big data analysis 21.3 Big data analysis algorithm 31.4 Big data analysis tool 61.5 Summary of this chapter 9Chapter 2 Data Feature Algorithm Analysis 102.1 Data distribution analysis 102.1.1 Determination of centralized trend of data distribution characteristics 102.1.2 Determination of dispersion of data distribution characteristics 152.1.3 Measurement of skewness and kurtosis of data distribution characteristics 192.2 Data correlation analysis 212.2.1 Data relationship212.2.2 Main contents of data correlation analysis 242.2.3 Determination of correlation 242.3 Data clustering analysis 262.3.1 Definition of cluster analysis 262.3.2 Cluster type 272.3.3 Clustering application 292.4 Principal component analysis of data 292.4.1 Principle and model of principal component analysis 302.4.2 Geometric interpretation of data principal component analysis 312.4.3 Export of data principal components322.4.4 Prove that the variance of principal components decreases in turn 342.4.5 Calculation of data principal component analysis 352.5 Data dynamic analysis 362.6 Data visualization 402.7 Summary of this chapter 42Chapter 3 Big Data Analysis Tools: NumPy 433.1 Introduction to NumPy 433.2 NumPy Environment Installation Configuration 443.3 ndarray objects453.4 Data types473.5 Array properties 493.6 Array creation routine523.7 Slicing and index573.8 Broadcasting 603.9 Array Operation and Iteration613.10 Bit Operations and String Functions873.11 Mathematical operation function913.12 Arithmetic operation933.13 Statistical function973.14 Sorting, searching and counting functions 1013.15 Byte exchange 1043.16 Replicas and views 1053.17 Matrix Library1073.18 Linear Algebra Module1093.19 Matplotlib library1123.20 Matplotlib drawing histograms 1143.21 IO file operation1163.22 NumPy example: GPS positioning 1173.23 Summary of this chapter 120Chapter 4 Big Data Analysis Tool: SciPy 1214.1 Introduction to SciPy 1214.2 File input and output: SciPy.io 1224.3 Special function: SciPy.special 1234.4 Linear algebra operation: SciPy.linalg 1244.5 Fast Fourier transform: sipy.fftpack 1244.6 Optimizer: SciPy.optimize 1254.7 Statistical tool: SciPy.stats 1264.8 SciPy Instance1274.8.1 Least squares fitting 1274.8.2 Minimum value of function1284.9 Summary of this Chapter130Chapter 5 Big Data Analysis Tool: Matplotlib 1315.1 Basic drawing 1315.2 Image, sub area, sub image, scale 1375.3 Other kinds of drawings1405.4 Summary of this chapter 147Chapter 6 Big Data Analysis Tool: Pandas 1486.1 Pandas Series1486.2 Pandas data frame1516.3 Pandas faceplate1556.4 Pandas Quick Start 1586.5 Summary of this chapter 172Chapter 7 Big Data Analysis Tools: Statsmodes and Gensim 1737.1 Statsmodels 1737.1.1 Statsmodes statistical database1737.1.2 Overview of Statsmodes typical fitting model1757.1.3 Statsmodes Example 1767.2 Gensim 1787.2.1 Basic Concept1787.2.2 Preprocessing of training corpus 1797.2.3 Transformation of subject vector 1807.2.4 Calculation of document similarity 1817.3 Summary of this chapter 182Chapter 8 Big Data Analysis Algorithms and Examples 1838.1 Descriptive statistic1838.2 Hypothesis test1888.3 Reliability analysis 1928.4 Analysis of contingency table1958.5 Correlation analysis 1968.6 Analysis of Variance1988.6.1 Single factor ANOVA 1998.6.2 Multivariate ANOVA 2018.7 Regression analysis2038.8 Cluster analysis 2078.9 Discriminant analysis2128.10 Principal component analysis 2168.11 Factor analysis 2188.12 Time series analysis2218.13 Survival analysis2248.14 Typical correlation analysis2458.15 RoC analysis2508.16 Distance analysis 2558.17 Correspondence analysis2648.18 Decision tree analysis2658.19 Neural Networks - Deep Learning 2718.19.1 Basic model of deep learning 2718.19.2 Examples of news classification2758.20 Monte Carlo Simulation 2808.20.1 Basic model of Monte Carlo simulation 2818.20.2 Example of Monte Carlo simulation calculation of call options2818.21 Association rule2878.21.1 Concept of association rule2888.21.2 Apriori algorithm and example 2898.21.3 FP tree frequency set algorithm2928.22 Uplift Modeling 3018.23 Integration method3068.24 Abnormality detection3118.25 Text mining 3158.26 Boosting algorithm (lifting method and Gradient Boosting) 3228.27 Summary of this chapter 325References 326
Introduction: In the era of artificial intelligence, Python is undoubtedly the hottest programming language.Some people boast that it is powerful and easy to learn. Others say that its learning curve is not so steep, but more people find that Python is easy to get started but not easy to master.Have you ever been so foolish that you couldn't distinguish the usage of "list", "tuple", "dictionary", "set", etc., and even tried to use indexing in the set?Have you ever worked hard on the idea of object-oriented, but when you were asked to design a slightly more complex system, you were helpless?Have you ever envied that others can skillfully use decorators, generators and other advanced operations, but when you write code, you are afraid to deal with such boundary conditions as exception throwing and insufficient memory?It can be seen that if you want to master this language, you must really understand the knowledge concepts, such as properly deepening the understanding from the source code level, then being familiar with the actual engineering applications, and independently completing the project development.In this way, you can become a real language expert.In this column, Jing Xiao will take you to learn Python from the perspective of engineering.Based on the latest version 3.7 of Python, the column focuses on the combination of language knowledge and engineering applications, including a large number of exclusive interpretations and practical work cases.The content is difficult and easy to take into account. It can not only help you consolidate the core foundation, but also teach you various advanced operations, so that you can gradually and systematically master the language Python.The column is divided into four modules according to the advanced difficulty.The first two parts are mainly the basic part and advanced part of Python.Apart from the necessary concepts and operation explanations, both the basic and advanced chapters emphasize the key, difficult and error prone points in learning, and start from different dimensions such as performance analysis and practical application examples, so that you can easily understand and master them.The third part is the specification chapter, which teaches you to write high-quality Python programs by explaining specific programming skills such as reasonable code decomposition, using assert, and writing unit tests.The fourth part is the practical part. This part will take you to connect the Python knowledge you learned earlier through the development of the quantitative trading system project, and add a lot of practical experience and skills, so that you can obtain quality improvement in the development of independent projects.
Points: 1Type: Technical documentUploader:liuguangjun196Upload time: March 29, 2023
Introduction: Python Project Case Development from Introduction to Practice - Crawlers, Games and Machine Learning was published by Tsinghua University Press in 2019. This book takes Python 3.5 as the programming environment, starts with basic programming ideas, and gradually launches Python language teaching. It is a programming book aimed at programming learners.This book uses cases to drive the explanation of knowledge points, and decomposes Python knowledge points into different cases. Each case has its own focus. At the same time, it shows the design idea and design concept of the actual project, so that readers can draw inferences from one instance.The cases in this book are practical, such as campus network search engines, small translators, and crawling Baidu pictures. With slight modifications, these crawler cases can be applied to actual projects;There are also cases of WeChat robot development through WeChat communication protocol, text classification of machine learning, handwriting recognition based on convolutional neural network, etc;In addition, there are some familiar game cases, such as Lianliankan, Pushbox, Chinese chess, online gobang, two person mahjong, figure puzzles and airplane fights.Through this book, readers will master Python programming technology and skills, learn object-oriented design methods, and understand all relevant content of program design.The contents of this book are as follows:Chapter 1 Python Basics 11.1 Introduction to Python 11.2 Python Syntax Basics 21.2.1 Python data type 21.2.2 Sequence data structure41.2.3 Python Control Statement121.2.4 Python functions and modules181.3 Python object-oriented design 221.3.1 Definition and use class 221.3.2 Constructor231.3.3 Destructor241.3.4 Instance Properties and Class Properties241.3.5 Private Members and Public Members 251.3.6 Method 261.3.7 Inheritance of Classes 271.3.8 Polymorphism291.3.9 Object oriented application case - poker licensing program 311.4 Python graphical interface design 341.4.1 Creating Windows Windows351.4.2 Geometric Layout Manager351.4.3 Tkinter assembly391.4.4 Tkinter font 491.4.5 Python Event Handling 511.4.6 Graphic interface design application case - developing number guessing game 551.5 Use of Python Files571.5.1 Opening/Creating a Document571.5.2 Reading Text File591.5.3 Writing Text Files601.5.4 Intra file movement621.5.5 Closing of documents 631.5.6 Reading/writing binary files 641.6 Third party library of Python 66Chapter 2 Sequence Application - Word Guessing Game 672.1 Introduction to the function of guessing words672.2 Program design thinking 672.3 Key technology random module682.4 Procedure of programming 71Chapter 3 Database Application - Intelligence Question and Answer Test 733.1 Introduction to intelligence question and answer test function733.2 Program design thinking 733.3 Key technologies743.3.1 Steps to access the database743.3.2 Creating Databases and Tables753.3.3 Insertion, update and deletion of database763.3.4 Querying Database Tables773.3.5 Example of database use - student address book 773.4 Steps of programming 803.4.1 Generating Test Question Library803.4.2 Reading test question information813.4.3 Interface and logic design81Chapter 4 Calling Baidu API Application - Little Translator 834.1 Introduction to functions of mini translator 834.2 Program design thinking 834.3 Key technologies844.3.1 Introduction to urllib library844.3.2 Basic use of urllib library844.4 Steps of program design 904.4.1 Design Interface 904.4.2 Using Baidu Translation Open Platform API 90Chapter 5 Crawler Application - Campus Web Search Engine 955.1 Function analysis of campus network search engine 955.2 Campus network search engine system design 955.3 Key technologies985.3.1 Regular Expression985.3.2 Chinese word segment1035.3.3 Installing and using jieba 1035.3.4 Adding a custom dictionary to jieba 1045.3.5 Keyword extraction for text categorization 1055.3.6 deque 1065.4 Procedure design 1075.4.1 Information acquisition module - implementation of web crawler 1075.4.2 Index module - building inverted thesaurus 1115.4.3 Web page ranking and search module113Chapter 6 Crawler Application - Grab Baidu Pictures 1166.1 Program function introduction1166.2 Program design thinking 1166.3 Key technologies1176.3.1 Downloading image files locally 1176.3.2 Crawl the pictures in the designated webpage 1176.3.3 Beautiful Soup Library Overview 1196.3.4 Using Beautiful Soup Library Operation to Parse HTML Document Tree 1216.3.5 Use of requests library1256.4 Steps of program design1336.4.1 Analyzing Web Page Source Code and Web Page Structure 1336.4.2 Design code136Chapter 7 itchat application WeChat robot 1397.1 Introduction to itchat function1397.2 Thinking of program design1407.3 Key technologies1407.3.1 Installing itchat 1407.3.2 WeChat login of itchat 1407.3.3 Message types of itchat 1417.3.4 itchat reply message1437.3.5 Obtaining an account by itchat 1457.3.6 Some simple applications of itchat 1477.3.7 Python calls Turing robot API to realize simple human-computer interaction 1507.4 Steps of program design1527.5 Developing a message synchronization robot 153Chapter 8 WeChat Web Protocol Application - WeChat Robot 1558.1 Introduction to Robot Functions on WeChat Webpage 1558.2 Design idea of WeChat web robot 1558.2.1 Analysis WeChat web page API 1558.2.2 API Summary1588.2.3 Other descriptions1648.3 Steps of programming 1668.3.1 Operation process of WeChat webpage version 1668.3.2 Program directory 1678.3.3 Implementation of WeChat web page version running code 1678.4 Extended function1708.4.1 Automatic response1708.4.2 Mass messaging, regular messaging, and friend status detection1738.4.3 Automatically invite friends to join group chat175Chapter 9 Image Processing - Generating QR Code and Verification Code 1789.1 Introduction to QR code 1789.2 Key technologies of two-dimensional code generation and analysis 1799.2.1 Use of qrcode library1799.2.2 Use of PIL library1829.3 Steps of 2D code generation and parsing program1849.3.1 Generating QR code with icon 1849.3.2 Python parsing QR code picture 1869.4 Generating the verification code image with Python 186Chapter 10 Puzzle Games - Lianliankan Games 18910.1 Lianliankan Introduction 18910.2 Ideas of program design19010.3 Key technologies20010.3.1 Graphic drawing - Tinker's Canvas component 20010.3.2 Graphic objects on Canvas 20010.4 Steps of programming 210Chapter 11 Puzzle Game - Box Pushing Game 21511.1 Introduction to Box Pushing Games 21511.2 Thinking of program design21611.3 Key technologies21711.4 Steps of programming 218Chapter 12 Entertainment Games - Two player Mahjong Game 22412.1 Introduction to Mahjong Games 22412.1.1 Mahjong terminologies22412.1.2 Number of plates 22412.2 Design ideas for two player mahjong game 22512.2.1 Material picture22512.2.2 Logic implementation of the game 22612.2.3 Judgment of touching/eating cards 22612.2.4 Heel Algorithm 22712.2.5 Realize computer intelligent licensing 23112.3 Key technologies23312.3.1 Sound playing23312.3.2 Components Returning to Corresponding Position23312.3.3 Sorting the list of saved mahjong tiles 23412.4 Steps of Two player Mahjong Game Design 23512.4.1 Design Mahjong Tiles 23512.4.2 Designing the main game program237Chapter 13 Network Programming Case -- Online Chat Program Based on TCP 24713.1 Introduction to TCP based online chat program24713.2 Key technologies24713.2.1 Internet TCP/IP Protocol24713.2.2 IP Protocol and Port24813.2.3 TCP Protocol and UDP Protocol24913.2.4 Socket 24913.2.5 Multi thread programming25413.3 Steps of online chat program25613.3.1 Server side of online chat program25613.3.2 Clients of online chat program259Chapter 14 Network Communication Case -- Network Gobang Based on UDPGames 26314.1 Introduction to online gobang games 26314.2 Design Ideas of Gobang Games 26414.3 Key technologies26714.3.1 UDP programming26714.3.2 Customizing the communication protocol of online gobang games 26914.4 Steps of online gobang game programming 27114.4.1 Steps of server-side programming 27114.4.2 Steps of client programming 276Chapter 15 Puzzle Game - Chinese Chess 28115.1 Introduction to Chinese Chess 28115.2 Key technologies28215.3 Design ideas of Chinese chess 28415.4 Steps of Chinese Chess Implementation287Chapter 16 Entertainment Games - Figure Puzzle Games 29716.1 Introduction to character jigsaw games 29716.2 Ideas of program design29816.3 Key technologies29816.3.1 Copying and pasting image area29816.3.2 Adjusting dimensions and rotation29816.3.3 Converting to gray-scale image29916.3.4 Operating pixels 30016.4 Procedure design 30016.4.1 Python processing image cutting 30016.4.2 Logical implementation of the game 302Chapter 17 Game Design Based on Pygame 30617.1 Pygame Basics 30617.1.1 Installing Pygame Library30617.1.2 Pygame module30617.2 Use of Pygame 30917.2.1 The main process of Pygame game development30917.2.2 Image/graphic rendering of Pygame 31117.2.3 Processing of keyboard and mouse events in Pygame 31417.2.4 Use of Pygame typeface 31917.2.5 Sound playback of Pygame 32017.2.6 Use of Pygame genies32117.3 Design of Snake Game Based on Pygame 32617.4 Design of aircraft war game based on Pygame 33317.4.1 Game roles33317.4.2 Display of game interface33617.4.3 Logic implementation of the game 338Chapter 18 Machine learning case - based on naive Bayesian algorithmText classification 34318.1 Introduction to text classification34318.2 Ideas of program design34318.3 Key technologies34418.3.1 Theoretical basis of Bayesian algorithm34418.3.2 Naive Bayesian classification34618.3.3 Text classification using Python 34818.4 Steps of programming 34818.4.1 Collecting training data34818.4.2 Preparing data34918.4.3 Analysis data34918.4.4 Training algorithm35018.4.5 Test algorithm and improve 35318.4.6 Text classification using algorithm35418.5 Using Naive Bayesian Classification Algorithm to Filter Spam 35518.5.1 Collecting training data35518.5.2 Parsing Text Files into Word Vector35618.5.3 Email classification using naive Bayesian algorithm35718.5.4 Improved algorithm35918.6 Text classification using Scikit-Learn library 36018.6.1 Common classes and functions for text classification36018.6.2 Case Realization363Chapter 19 Case of Deep Learning -- Convolution Neural Network BasedHandwriting recognition 36619.1 Handwriting recognition case requirements36619.2 Concept and key technologies of deep learning 36619.2.1 Neural network model36619.2.2 Convolution neural network for deep learning 36719.3 Python deep learning library Keras 37219.3.1 Installation of Keras 37219.3.2 Network layer of Keras 37219.3.3 Building Neural Networks with Keras 37519.4 Ideas of program design37619.5 Steps of programming 37719.5.1 MNIST Dataset37719.5.2 Realization of handwriting recognition case37819.5.3 Predicting Your Own Handwritten Image382Chapter 20 Ciyun Actual Battle - Climbing Douban Film Review to Generate Ciyun 38320.1 Function introduction38320.2 Ideas of program design38420.3 Key technologies38520.3.1 Installing WordCloud 38520.3.2 Using WordCloud 38520.4 Steps of programming 389References 397
Introduction: This book mainly takes the domestic A-share market as an example and uses the third-party quantitative trading platform to describe the application of common machine learning algorithms such as KNN, linear model, decision tree, support vector machine and naive Bayes in trading strategies. At the same time, it shows how to back test strategies so that readers can effectively evaluate their own strategies.In addition, the book also explains the development trend of natural language processing (NLP) technology in the field of quantitative trading, and introduces readers to the forward-looking application of multi-layer perceptron, convolutional neural network, and short-term memory network in quantitative trading using the popular in-depth learning technology.This book does not start with the basic syntax of Python, and it just skims over the traditional trading strategies, directly bringing readers into the world of machine learning.This book is suitable for readers who have some knowledge of Python and are interested in quantitative trading.VIIcatalogChapter 1 Xiaowa's Story - From scratch1.1 How to relieve worries? "Little rich" is also OK 11.1.1 Those years, those transactions 21.1.2 Automated trading and high-frequency trading 21.1.3 Factor investment rises quietly 31.2 The rise of machine learning 41.2.1 Quantitative investment is flourishing 41.2.2 No data is allowed 51.2.3 Trading strategy and alpha factor 51.3 If you want to be rich, you should first configure the warehouse 61.3.1 Downloading and installing Anaconda 61.3.2 Basic usage of Jupyter Notebook 81.3.3 Practice with real stock data 111.4 Summary 15Chapter 2 Is Xiaowa's Strategy Reliable Backtesting and Classic Strategies2.1 Simple back test of small watt strategy 162.1.1 Download Data and Create Transaction Signal162.1.2 Simple Backtesting of Trading Strategies 182.1.3 What you need to know about backtesting 202.2 Moving average strategy of classic strategy 212.2.1 Single moving average index212.2.2 Implementation of dual moving average strategy 232.2.3 Backtesting the Dual Moving Average Strategy 262.3 Classic Strategy: Turtle Strategy 282.3.1 Using Turtle Strategy to Generate Trading Signals 282.3.2 Place an order according to the trading signal and position 292.3.3 Backtesting the Turtle Strategy 312.4 Summary 34Chapter 3 Here Comes AI - Simple Application of Machine Learning in Trading3.1 Basic concepts of machine learning 353.1.1 Supervised learning and unsupervised learning 35Simply explain Python quantitative trading practiceVIII3.1.2 Classification and regression 373.1.3 Evaluation of model performance373.2 Basic usage of machine learning tools 373.2.1 Basic principle of KNN algorithm 383.2.2 KNN algorithm for classification383.2.3 KNN algorithm for regression 433.3 Simple trading strategy based on machine learning 473.3.1 Obtaining stock data 473.3.2 Creating Transaction Conditions493.3.3 Using Classification Algorithms to Develop Trading Strategies 503.4 Summary 54Chapter 4 Multi source data - with the help of quantitative trading platform4.1 The data is insufficient, and the platform makes up 554.1.1 Selecting quantitative trading platform564.1.2 Research environment of quantitative trading platform574.1.3 Running code in the research environment584.2 Screening stocks with financial data 594.2.1 Overview of stock acquisition 604.2.2 Obtaining financial data of stock 624.2.3 Stock selection through financial indicators 644.3 Who is the "Big Boss" 654.3.1 Shareholders Found 664.3.2 Whether major shareholders increased or decreased their shareholdings 674.3.3 Net inflow or outflow of funds 694.4 Summary 71Chapter 5 Here Comes the Factor - Basic Principle and Usage5.1 Understanding the "Wahl factor" 725.1.1 Obtaining main fund flow data 735.1.2 Simple feature engineering745.1.3 Calculation of "Wahl factor" 755.1.4 Training the model with data added with "Wahl factor" 765.1.5 What can "factors" do 775.2 How to choose stocks?Factor to help 785.2.1 Determining the stock pool 785.2.2 Obtain all indexes of Shanghai and Shenzhen Stock Exchanges795.2.3 Acquire the market value factor of the stock 805.2.4 Cash flow factors for acquiring stocks 815.2.5 Obtain the net interest rate factor of the stock 825.2.6 Growth rate factor of net profit obtained from stock 835.3 "Package" many factors 845.3.1 Saving four factors into a DataFrame 845.3.2 Extraction of principal components by PCA855.3.3 Finding stocks with principal component values865.4 Summary 87catalogIXChapter 6 Is the factor easy to use - there are some things you need to know6.1 Obtaining factor values for portfolios 886.1.1 Establishing a Portfolio and Setting a Date886.1.2 Obtaining an Emotional Factor 906.1.3 Obtaining all factor analysis results916.2 Factor income analysis 926.2.1 Quantile statistics of factors 926.2.2 Cumulative income of factor weighted multi empty portfolio 946.2.3 Gains from multi quantile and short small quantile 966.2.4 Quantile cumulative income comparison 976.3 Factor IC Analysis 986.3.1 Overview of factor IC analysis996.3.2 Factor IC Time Series Chart996.3.3 Factor IC normal distribution Q-Q chart and monthly mean value 1016.4 Factor turnover, factor autocorrelation and factor predictionMeasurement capability analysis 1026.4.1 Factor turnover rate analysis 1036.4.2 Factor Autocorrelation Analysis 1046.4.3 Factor prediction capability analysis 1066.5 Summary 107Chapter 7 When Factors Meet Linear Models7.1 What is a linear model1087.1.1 Preparing data for demonstration 1087.1.2 Try simple linear regression 1107.1.3 Using the Regularized Linear Model1137.2 Making Trading Strategies with Linear Models 1157.2.1 Preparation factor1157.2.2 Training model1177.2.3 Stock selection based on model prediction 1187.3 Can we make money 1197.3.1 Strategy backtesting function of the platform 1207.3.2 Writing research results into strategies 1217.3.3 Backtesting 1247.4 Summary 126Chapter 8 Factor Encounters Decision Tree and Random Forest8.1 What are decision trees and random forests 1278.1.1 Data samples not applicable to linear model1278.1.2 Usage and principle of decision tree 1298.1.3 Usage and principle of random forest 1308.2 What factors are important? The decision tree can tell you 1328.2.1 Multiple source factor1328.2.2 Setting goals and training model1358.2.3 Which factors are important 1378.3 Formulated with important factors and random forestsStrategy 138Simply explain Python quantitative trading practice8.3.1 Initialization of backtesting function1388.3.2 Preparation before disc1398.3.3 Machine learning part of the strategy1418.3.4 List of defining buying and selling stocks 1428.3.5 Defining Buying and Selling Operations 1448.3.6 Back testing the strategy 1458.4 Summary 146Chapter 9 Factors Encounter Support Vector Machines9.1 What is support vector machine 1479.1.1 Basic principle of support vector machine1479.1.2 The linear kernel is sometimes "in a hurry" 1499.1.3 RBF kernel "shining" 1509.2 Dynamic factor selection strategy1529.2.1 Setting the backtesting environment1529.2.2 Preparation before opening 1539.2.3 Machine learning part1559.2.4 Buying and selling operations1579.3 Backtesting details of strategy1589.3.1 Overview of Strategic Benefits 1599.3.2 Details of strategic transactions 1599.3.3 Position and income detail1619.4 Simulated transactions using strategies 1629.4.1 Simulated transaction 1639.4.2 Viewing simulated transaction details1649.4.3 Position and order placement of simulated transaction 1659.5 Summary 166Chapter 10 First Knowledge of Natural Language Processing Technology10.1 Is our idea reliable 16710.1.1 Thinking about Several Questions 16710.1.2 Refer to the practice of "Big Boss" 16810.1.3 Having said so much, what is NLP 16910.2 Obtaining text data and simple cleaning 17010.2.1 Obtaining news broadcast text data17010.2.2 Simple cleaning of text data17210.3 Chinese word segmentation, "stutter" to help 17310.3.1 Using "stutter" to participle 17410.3.2 Using "stutter" for list participation17410.3.3 Creating a Deactivated Thesaurus 17510.3.4 Removing the stop words in the text17610.3.5 Using "stutter" to extract keywords 17810.4 Summary 180Chapter 11 news text vectorization and topic modeling11.1 Let the machine "read" the news18111.1.1 Preparing text data181catalog11.1.2 Convert text toVector 18311.1.3 Use TfidfVectorizer to convert text toVector 18511.2 Let the machine tell us what the news says 18611.2.1 What is topic modeling 18611.2.2 What is LDA model18711.3 Topic modeling practical18811.3.1 Loading Data and Word Segmentation18811.3.2 Merging and saving word segmentation results19011.3.3 Using LDA for topic modeling19111.3.4 Improving the model19211.4 Summary 194Chapter 12 Emotional Analysis of Stock Evaluation Data12.1 Does the machine understand our emotions 19512.1.1 Understanding the Classified Corpus 19612.1.2 Uploading files to quantitative trading platform19712.2 Making data sets from corpus 19812.2.1 Storing Positive Emotional Corpus as a List19812.2.2 Storing Negative Emotion Corpus as List20012.2.3 Labeling Data 20112.2.4 Combining Positive and Negative Emotional Corpus 20212.3 The grand launch of "naive Bayes" 20312.3.1 What is "naive Bayes" 20412.3.2 Preparing data for Bayesian model20512.3.3 Start training Bayesian models and evaluate their performance20612.4 Summary 208Chapter 13 We are also in the limelight of deep learning13.1 Preparation before starting study20913.1.1 Flip over the toolbox to see what there is 21013.1.2 Preparing data for neural network21113.2 Using Keras to preprocess text21313.2.1 Using Tokenizer to Extract Features 21313.2.2 Converting Text to Sequence21413.2.3 Filled sequence and transformation matrixes 21613.3 Building a Simple Neural Network Using Keras 21713.3.1 First, "roll out" a multi-layer perceptron 21713.3.2 Talk about the principle of multi-layer perceptron 21813.3.3 Let's talk about activation function22013.3.4 What does the Dropout layer do 22113.3.5 Train to see the effect 22213.4 Summary 224Simply explain Python quantitative trading practiceChapter 14 goes further - CNN and LSTM14.1 Start to "roll" a convoluted nerveNetwork 22514.1.1 Preparing Libraries and Datasets 22514.1.2 Processing Data and Modeling 22714.2 Detailed explanation of convolutional neural network model22914.2.1 What is the embedded layer used for 23014.2.2 What is the roll up layer used for 23114.2.3 What is the pool layer used for 23314.2.4 Viewing the effect of training model23414.3 Long and short-term memory network23614.3.1 Building a simple short-term memory network23614.3.2 About long-term and short-term memory network23714.3.3 Training model and evaluation23814.3.4 Save the model and call 24014.4 Summary 241Chapter 15 is written in the back - Xiaowa's journey15.1 Can we get rich overnight 24215.1.1 It is a good idea to use the third-party quantification platformYima24315.1.2 Is machine learning useful or not 24315.1.3 "Hang" in the "tree" of A-shareGo 24415.2 What to do in the future24515.2.1 Learn some database knowledge24515.2.2 Look at different investment objects24715.2.3 Opening the International Horizon 24915.3 Summary 252
Introduction: Quantitative Financial Investment and Its Python Application includes: (1) Python working environment for quantitative financial investment;(2) Basic knowledge and programming of Python;(3) Quantitative financial investment package Python NumPy application;(4) Quantitative financial investment package Python CiPy application;(5) Basic data structure of quantitative financial investment package Python Pandas;(6) Application of quantitative financial investment package Python Pandas in financial data processing;(7) Financial time series analysis and its Python application;(8) Chinese stock market analysis and its Python application;(9) Machine learning neural network algorithm and its Python application;(10) Machine learning support vector machine SVM and its Python application;(11) Python application of European option pricing;(12) Python application of function interpolation;(13) Python application of option pricing binary tree algorithm;(14) Python application of explicit difference method for partial differential equations;(15) Python application of implicit difference method for partial differential equations;(16) Black Scholes partial differential equation implicitChapter 1 Quantitative Financial Investment Platform and Python Working Environment1.1 Overview of domestic and foreign quantitative financial investment platforms1.2 Interface of ore optimization platform1.3 Services provided by the optimized mining platform1.4 Notebook function of ore optimization platform1.5 Python package supported by optimized mining platform1.6 Downloading Python1.7 Python Installation1.8 Python startup and exitExercisesChapter 2 Two Basic Python Operations and Programming Basics2.1 Two basic operations of Python2.2 Python container2.3 Python functions2.4 Python conditions and loops2.5 Python classes and objectsExercisesChapter 3 Application of NumPy in Quantitative Financial Investment Analysis3.1 Overview of NumPy3.2 Preliminary NumPy object: array3.3 Creating Arrays3.4 Operation of array and matrix3.5 Accessing arrays and matrix elements3.6 Matrix operation3.7 Missing values3.8 NumPy application of unitary linear regression analysisExercisesChapter 4 Application of SciPy in Quantitative Financial Investment Analysis4.1 Overview of SciPy4.2 Statistical knowledge4.3 Optimizing knowledge4.3.1 Unconstrained optimization problems4.3.2 Constrained optimization problems4.3.3 Using CVXOPT to solve quadratic programming problemsExercisesChapter 5 Basic Data Structure of Pandas5.1 Introduction to pandas5.2 Pandas data structure: Series5.2.1 Creating Series5.2.2 Access to Series Data5.3 Pandas data structure: DataFrame5.3.1 Create DataFrame5.3.2DataFrame data accessExercisesChapter 6 Application of Pandas in Financial Data Processing6.1 How to create data structure6.2 Data viewing6.3 Data access and operation6.3.1 Re talk about data access6.3.2 Handling missing data6.3.3 Data operation6.4 Data visualizationExercisesChapter 7 Financial Time Series Analysis and Its Python Application7.1 Basic knowledge of time series analysis7.1.1 Concept and characteristics of time series7.1.2 Stability7.1.3 Correlation coefficient and autocorrelation function7.1.4 White noise series and linear time series7.2 Autoregression model7.2.1 Characteristic root and stationarity test of AR (p) model7.2.2AR (p) model order determination7.2.3 Model inspection7.2.4 Goodness of fit and prediction7.3 Moving average model and prediction7.3.1 Properties of MA (q) model7.3.2 Order determination of MA (q) model7.3.3 Modeling and forecasting7.4 Autoregressive moving average model and prediction7.4.1 Determining the order of ARMA (p, q) model7.4.2 Establishment and prediction of ARMA model7.5 ARIMA model and prediction7.5.1 Unit root inspection7.5.2 Determination of ARIMA (p, d, q) model order7.5.3 Establishment and prediction of ARIMA model7.6 Autoregressive conditional heteroscedasticity model ARCH and prediction7.6.1 Characteristics of volatility7.6.2 Basic principle of ARCH model7.6.3 Establishment and prediction of ARCH model7.7 Generalized autoregressive conditional heteroscedasticity model GARCH and volatility prediction7.7.1 Establishment of GARCH model7.7.2 Volatility forecastExercisesChapter 8 China Stock Market Analysis and Python Application8.1 Basic information of shares8.2 Analysis of stock return risk8.3 Monte Carlo method based on VaRExercisesChapter 9 Machine learning neural network algorithm and its Python application9.1 Topology of BP neural network9.2 Learning algorithm of BP neural network9.3 Learning procedure of BP neural network9.4 Python application of BP neural network algorithm for stock predictionExercisesChapter 10 Machine Learning Support Vector Machine and Its Python Application10.1 Machine learning support vector machine principle10.2 Application of machine learning support vector machineExercisesChapter 11 Python Application of European Option Pricing11.1 Python function of option pricing formula11.2 Using NumPy to Accelerate Batch Computing11.2.1 Mode of using cycle11.2.2 Calculation using NumPy vector11.3 Using SciPy for Simulation Calculation11.4 Calculation of implied volatilityExercisesChapter 12 Python Application of Function Interpolation12.1 How to use SciPy for function interpolation12.2 Application of Function Interpolation -- Construction of Option Volatility SurfaceExercisesChapter 13 Python Application of Option Pricing Binary Tree Algorithm13.1 Python description of binary tree algorithm13.2 Implementation of binary tree algorithm with object-oriented method13.2.1 Binary Tree Frame13.2.2 Binary Tree Type Description13.2.3 Reimbursement function13.2.4 Assembly13.3 Binary tree algorithm for American option pricingExercisesChapter 14 Python Application of Explicit Difference Method for Partial Differential Equations14.1 Heat conduction equation14.2 Explicit difference scheme14.3 Module assembly14.4 Conditional stability of explicit schemesExercisesChapter 15 Python Application of Implicit Difference Method for Partial Differential Equations15.1 Implicit difference scheme15.1.1 Matrix solution15.1.2 Implicit scheme solution15.2 Module assembly15.3 Using SciPy AccelerationExercisesChapter 16 Python Applications of Implicit Difference Methods for Black Scholes Merton Partial Differential Equations16.1 Proposition of the Initial Boundary Value Problem of Black Scholes Merton Partial Differential Variance16.2 Implicit difference method for partial differential equations16.3 Python Application Implementation16.4 Convergence testExercisesChapter 17 Preliminary quantitative financial investment of the excellent mining platform17.1 Quantitative financial investment basis17.2 Quantitative financial investment and its strategy17.3 Setting Initial Data17.4 Select Stock Pool17.5 Initialize the back test account17.6 Setting conditions for purchase and sale17.7 Combine into a complete quantitative strategyExercisesChapter 18 Python Application of Alpha Hedge Model18.1 Alpha hedging model18.2 "Three Swordsmen" on the excellent mining platform18.3 Example of hedging model of optimized mining platformExercisesChapter 19 Python Application of Alpha Quantitative Financial Investment Strategy under the Signal Framework19.1 Why Alpha Hedging Model19.2 Signal framework, the artifact of building alpha hedging model on the optimal mining platform19.3 How does a typical public fund team build its own alpha hedging model19.4 How to surpass a public fund team on the excellent mining platformExercisesChapter 20 Python Application of Quantitative Financial Portfolio Optimization20.1 Markowitz's basic theory of portfolio optimization20.2 Python application examples of portfolio optimization20.3 Python application of actual data for portfolio optimizationExercisesreference
Points: 1Type: Technical documentUploader:HyperplatinumUpload time: May 30, 2021
Introduction: Python Computer Vision Programming is the authoritative practice guide of computer vision programming. It explains the basic theory and algorithm relying on Python language, and analyzes object recognition, content-based image search, optical character recognition, optical flow method, tracking, 3D reconstruction, stereo imaging, augmented reality, pose estimation, panoramic creationImage segmentation, noise reduction, image grouping and other technologies.In addition, the exercises included in the book can also help readers consolidate and learn the knowledge of applied programming.Python Computer Vision Programming is suitable for students who have a certain foundation in programming and mathematics and want to understand the basic theories and algorithms of computer vision, as well as researchers and practitioners in computer science, signal processing, physics, applied mathematics and statistics, neurophysiology, cognitive science and other fields.
Introduction: In today's data driven world, machine learning is becoming more and more popular.It has been widely used in different fields, such as search engines, robots, driverless cars and so on.This book first introduces the basic knowledge of machine learning through practical cases, and then introduces some slightly complex machine learning algorithms, such as support vector machines, pole end random forests, hidden Markov models, conditional random fields, deep neural networks, and so on.This book is for Python programmers who want to develop applications with machine learning algorithms.It is suitable for Python beginners to read, but familiarity with Python programming methods is beneficial to experience sample code.Chapter 1 Supervised Learningone1.1 Introductionone1.2 Data pre-processing technologytwo1.2.1 Preparationtwo1.2.2 Detailed stepstwo1.3 Marking coding methodfour1.4 Creating a Linear Regressionsix1.4.1 Preparationsix1.4.2 Detailed stepsseven1.5 Calculation of regression accuracynine1.5.1 Preparationnine1.5.2 Detailed stepsten1.6 Saving Model Dataten1.7 Creating Ridge Regressioneleven1.7.1 Preparationeleven1.7.2 Detailed stepstwelve1.8 Creating Polynomial Regressionthirteen1.8.1 Preparationthirteen1.8.2 Detailed stepsfourteen1.9 Estimating the House Pricefifteen1.9.1 Preparationfifteen1.9.2 Detailed stepssixteen1.10 Calculate the relative importance of the characteristicsseventeen1.11 Evaluate the demand distribution of shared bicyclesnineteen1.11.1 Preparationnineteen1.11.2 Detailed stepsnineteen1.11.3 Moretwenty-oneChapter 2 Creating Classifierstwenty-four2.1 Introductiontwenty-four2.2 Building a simple classifiertwenty-five2.2.1 Detailed stepstwenty-five2.2.2 Moretwenty-seven2.3 Establishing a Logical Regression Classifiertwenty-seven2.4 Building a naive Bayesian classifierthirty-one2.5 Split the data set into training set and test setthirty-two2.6 Verifying model accuracy by cross validationthirty-three2.6.1 Preparationthirty-four2.6.2 Detailed stepsthirty-four2.7 Visualization of confusion matrixthirty-five2.8 Extraction performance reportthirty-seven2.9 Evaluation of quality based on vehicle characteristicsthirty-eight2.9.1 Preparationthirty-eight2.9.2 Detailed stepsthirty-eight2.10 Generating the validation curveforty2.11 Generating the Learning Curveforty-three2.12 Estimating income classforty-fiveChapter 3 Forecast Modelingforty-eight3.1 Introductionforty-eight3.2 Using SVM to establish a linear classifierforty-nine3.2.1 Preparationsforty-nine3.2.2 Detailed stepsfifty3.3 Building a Nonlinear Classifier with SVMfifty-three3.4 Solving the Problem of Unbalanced Types and Numbersfifty-five3.5 Extracting Confidencefifty-eight3.6 Finding the optimal super parametersixty3.7 Establishing an Event Predictorsixty-two3.7.1 Preparationsixty-two3.7.2 Detailed stepssixty-two3.8 Estimated Traffic Flowsixty-four3.8.1 Preparationsixty-four3.8.2 Detailed stepssixty-fourChapter 4 Unsupervised Learning Clusteringsixty-seven4.1 Introductionsixty-seven4.2 Clustering data with k-means algorithmsixty-seven4.3 Compressing pictures with vector quantizationseventy4.4 Establishing the mean shift clustering modelseventy-four4.5 Data grouping by agglomerative hierarchical clusteringseventy-six4.6 Evaluation of clustering effect of clustering algorithmseventy-nine4.7 Using DBSCAN algorithm to automatically estimate the number of clusterseighty-two4.8 Exploring the Patterns of Stock Dataeighty-six4.9 Establishing a Customer Segmentation Modeleighty-eightChapter 5 Building a Recommendation Engineninety-one5.1 Introductionninety-one5.2 Building Function Combinations for Data Processingninety-two5.3 Building a machine learning pipelineninety-three5.3.1 Detailed stepsninety-three5.3.2 Operating principleninety-five5.4 Finding the Nearest Neighborninety-five5.5 Building a KNN classifierninety-eight5.5.1 Detailed stepsninety-eight5.5.2 Working principleone hundred and two5.6 Building a KNN Regressionone hundred and two5.6.1 Detailed stepsone hundred and two5.6.2 Operating principleone hundred and four5.7 Calculation of Euclidean distance fractionone hundred and five5.8 Calculation of Pearson correlation coefficientone hundred and six5.9 Looking for similar users in the data setone hundred and eight5.10 Film recommendationone hundred and nineChapter 6 Analyzing Text Dataone hundred and twelve6.1 Introductionone hundred and twelve6.2 Preprocessing data with tag parsing methodone hundred and thirteen6.3 Stemming Text Dataone hundred and fourteen6.3.1 Detailed stepsone hundred and fourteen6.3.2 Working principleone hundred and fifteen6.4 Restore the basic form of the text with the method of word form restorationone hundred and sixteen6.5 Text division by block methodone hundred and seventeen6.6 Creating a Word Bag Modelone hundred and eighteen6.6.1 Detailed stepsone hundred and eighteen6.6.2 Operating principleone hundred and twenty6.7 Creating a Text Classifierone hundred and twenty-one6.7.1 Detailed stepsone hundred and twenty-one6.7.2 Operating principleone hundred and twenty-three6.8 Identification of genderone hundred and twenty-four6.9 Analyzing Sentence Sentences' Emotionsone hundred and twenty-five6.9.1 Detailed stepsone hundred and twenty-six6.9.2 Operating principleone hundred and twenty-eight6.10 Pattern of text recognition by topic modelingone hundred and twenty-eight6.10.1 Detailed stepsone hundred and twenty-eight6.10.2 Operating principleone hundred and thirty-oneChapter 7 Speech Recognitionone hundred and thirty-two7.1 Introductionone hundred and thirty-two7.2 Reading and Plotting Audio Dataone hundred and thirty-two7.3 Converting audio signal into frequency domainone hundred and thirty-four7.4 Generating audio signal by user-defined parametersone hundred and thirty-six7.5 Synthetic Musicone hundred and thirty-eight7.6 Extraction of frequency domain characteristicsone hundred and forty7.7 Creating Hidden Markov Modelsone hundred and forty-two7.8 Creating a speech recognizerone hundred and forty-threeChapter 8 Anatomy of Time Series and Time Series Dataone hundred and forty-seven8.1 Introductionone hundred and forty-seven8.2 Converting data to time series formatone hundred and forty-eight8.3 Split time series dataone hundred and fifty8.4 Operation time series dataone hundred and fifty-two8.5 Extracting statistics from time series dataone hundred and fifty-four8.6 Creating Hidden Markov Models for Sequence Dataone hundred and fifty-seven8.6.1 Preparationone hundred and fifty-eight8.6.2 Detailed stepsone hundred and fifty-eight8.7 Creating conditional random fields for sequential text dataone hundred and sixty-one8.7.1 Preparationone hundred and sixty-one8.7.2 Detailed stepsone hundred and sixty-one8.8 Using Hidden Markov Model to Analyze Stock Market Dataone hundred and sixty-fourChapter 9 Image Content Analysisone hundred and sixty-six9.1 Introductionone hundred and sixty-six9.2 Using OpenCV Pyhon to operate imagesone hundred and sixty-seven9.3 Detection sideone hundred and seventy9.4 Histogram Equalizationone hundred and seventy-four9.5 Detection of edges and cornersone hundred and seventy-six9.6 Detection of SIFT feature pointsone hundred and seventy-eight9.7 Create Star feature detectorone hundred and eighty9.8 Creating features using visual codebook and vector quantizationone hundred and eighty-two9.9 Training image classifier with extreme random forestone hundred and eighty-five9.10 Creating an Object Identifierone hundred and eighty-sevenChapter 10 Face Recognitionone hundred and eighty-nine10.1 Introductionone hundred and eighty-nine10.2 Collecting and processing video information from webcamsone hundred and eighty-nine10.3 Create a face recognizer with Haar cascadeone hundred and ninety-one10.4 Create an eye and nose detectorone hundred and ninety-three10.5 Principal component analysisone hundred and ninety-six10.6 Conduct nuclear principal component analysisone hundred and ninety-seven10.7 Blind source separationtwo hundred and one10.8 Create a human face recognizer using the local binary pattern histogramtwo hundred and fiveChapter 11 Deep Neural Networktwo hundred and ten11.1 Introductiontwo hundred and ten11.2 Creating a Sensortwo hundred and eleven11.3 Creating a Single Layer Neural Networktwo hundred and thirteen11.4 Creating a Deep Neural Networktwo hundred and sixteen11.5 Creating a vector quantizertwo hundred and nineteen11.6 Creating a Recurrent Neural Network for Sequence Data Analysistwo hundred and twenty-one11.7 Visualizing characters in the OCR databasetwo hundred and twenty-five11.8 Creating an optical character recognizer using neural networkstwo hundred and twenty-sixChapter 12 Visualization Datatwo hundred and thirty12.1 Introductiontwo hundred and thirty12.2 Draw 3D Scatter Charttwo hundred and thirty12.3 Drawing Bubble Charttwo hundred and thirty-two12.4 Drawing Dynamic Bubble Charttwo hundred and thirty-three12.5 Pie Charttwo hundred and thirty-five12.6 Time series data in date formattwo hundred and thirty-seven12.7 Draw Histogramtwo hundred and thirty-nine12.8 Visual Thermodynamic Diagramtwo hundred and forty-one12.9 Visual simulation of dynamic signalstwo hundred and forty-two
Introduction: This book is an introduction book for novices in machine learning. Starting from the establishment of the learning environment, it introduces Python knowledge and mathematical knowledge required for machine learning with pictures and text, and on this basis, it combines mathematical formulas, example programs, illustrations, etc. to exfoliate the regression and classification, neural network and deep learning algorithms and applications in supervised learningHandwritten digit recognition and unsupervised learning algorithms are introduced.This book has both graphics and code, as well as a detailed mathematical formula derivation process, which greatly reduces the learning threshold of machine learning. Even if you have not learned Python and have a poor mathematical foundation, you can understand it.Chapter 1 Preparation before Learning 11.1 About machine learning 21.1.1 Tips for learning machine learning 41.1.2 Classification of problems in machine learning 51.1.3 Structure of the book 61.2 Installing Python 71.3 Jupyter Notebook 111.3.1 Usage of Jupyter Notebook 111.3.2 Input Markdown Format Text141.3.3 Changing the file name 161.4 Installing Keras and TensorFlow 17Chapter 2 Python Basics 192.1 Four operations 202.1.1 Usage of four arithmetic operations202.1.2 Power operation 202.2 Variables 212.2.1 Calculation by variables 212.2.2 Naming of variables 212.3 Type 222.3.1 Type 222.3.2 Types of Inspection222.3.3 Strings 232.4 print statement242.4.1 Usage of print statement242.4.2 Method of displaying numeric value and string at the same time 1 242.4.3 Method of displaying numeric value and string at the same time 2 252.5 list (array variable) 262.5.1 List Usage 262.5.2 Two dimensional array 272.5.3 Creating Continuous Integer Array282.6 tuple (array) 292.6.1 Use of tuple 292.6.2 Reading Elements 292.6.3 Tuple 30 with length of 12.7 if statement312.7.1 Usage of if statement312.7.2 Comparison operators322.8 for statement 332.8.1 Usage of the for statement 332.8.2 Usage of enumerate332.9 Vectors342.9.1 Usage of NumPy342.9.2 Defining Vectors352.9.3 Reading elements362.9.4 Replacement elements362.9.5 Creating a Vector of Continuous Integers362.9.6 Precautions for array 372.10 Matrice382.10.1 Definition matrix 382.10.2 Size of matrix 382.10.3 Reading elements392.10.4 Replacement elements392.10.5 Generating an array of elements 0 and 1 392.10.6 Generating a matrix with random elements 402.10.7 Changing the size of the matrix 412.11 Four operations of matrix 412.11.1 Four operations of matrix 412.11.2 Scalar × matrix 422.11.3 Arithmetic functions422.11.4 Calculating Matrix Multiplication432.12 Slicing 432.13 Replacing Satisfactory Data452.14 help 462.15 Functions 472.15.1 Function usage 472.15.2 Parameters and return values472.16 Saving Files492.16.1 Saving an array type variable 492.16.2 Saving multiple array type variables 49Chapter 3 Data Visualization 513.1 Drawing two-dimensional graphics 523.1.1 Drawing Random Graphs 523.1.2 Format of code list533.1.3 Draw a cubic function f (x)=(x - 2) x (x+2) 533.1.4 Determining the drawing range 543.1.5 Drawing Graphs 553.1.6 Decorative graphics 553.1.7 Displaying Multiple Graphs in Parallel 583.2 Drawing 3D graphics 593.2.1 Functions with Two Variables 593.2.2 Color value: pcolor 603.2.3 Draw 3D graphics: surface 623.2.4 Contour drawing: contour 64Chapter 4 Mathematics in Machine Learning 674.1 Vectors684.1.1 What is a vector 684.1.2 Defining vectors with Python 694.1.3 Column vector representation 694.1.4 Representation of transposition 704.1.5 Adding and Subtracting 714.1.6 Scalar product 734.1.7 Inner product 744.1.8 Modulus of vector 754.2 Summing symbol 764.2.1 Deformation of mathematical formula with summation symbol 774.2.2 Summing by inner product 794.3 Accumulation symbol 794.4 Derivative804.4.1 Derivative of polynomial804.4.2 Deformation of mathematical formula with derivative sign 824.4.3 Derivative of composite function834.4.4 Derivative of composite function: chain rule 844.5 Partial derivative 854.5.1 What is a partial derivative 854.5.2 Graph of partial derivative 874.5.3 Plotting a Gradient 894.5.4 Partial derivative of compound function with multiple variables 914.5.5 Order of exchange sum and derivation 934.6 Matrix 954.6.1 What is matrix 954.6.2 Adding and Subtracting Matrices 974.6.3 Scalar Product 994.6.4 Product of matrix 1004.6.5 Identity matrix 1034.6.6 Inverse matrix 1054.6.7 Transposition 1074.6.8 Matrix and simultaneous equation 1094.6.9 Matrices and Mappings 1114.7 Exponential and logarithmic functions 1134.7.1 Index 1134.7.2 Logarithm 1154.7.3 Derivative of exponential function1184.7.4 Derivative of logarithmic function1204.7.5 Sigmaid function1214.7.6 Softmax function1234.7.7 Softmax function and Sigmoid function1274.7.8 Gaussian function1284.7.9 Two dimensional Gaussian function129Chapter 5 Supervised Learning: Return 1355.1 Linear model with one-dimensional input 1365.1.1 Linear model1385.1.2 Square error function 1395.1.3 Parameter calculation (gradient method) 1425.1.4 Analytical solution of linear model parameters1485.2 2D input plane model1525.2.1 Data representation method 1545.2.2 Plane model1555.2.3 Analytical solution of plane model parameters1575.3 D-dimensional linear regression model1595.3.1 D-dimensional linear regression model1605.3.2 Analytical solution of parameter1605.3.3 Extending to the plane not passing through the origin 1645.4 Linear basis function model1655.5 Overfitting problems 1715.6 Generation of new model1815.7 Selection of model1855.8 Summary 186Chapter 6 Supervised Learning: Classification 1896.1 Binary classification of one-dimensional input1906.1.1 Problem Setting1906.1.2 Using probability to represent classification 1946.1.3 Maximum likelihood estimation1966.1.4 Logical regression model1996.1.5 Cross entropy error 2016.1.6 Deduction of learning rule2056.1.7 Solve by gradient method2096.2 Binary classification of two-dimensional input2106.2.1 Problem Setting2106.2.2 Logical regression model2146.3 Three element classification of two-dimensional input2196.3.1 Triple classification logistic regression model2196.3.2 Cross entropy error2226.3.3 Solving by gradient method223Chapter 7 Neural Network and Deep Learning 2277.1 Neuron model2297.1.1 Neurocells2297.1.2 Neuron model2307.2 Neural network model2347.2.1 Two layer feedforward neural network2347.2.2 Implementation of two-layer feedforward neural network2377.2.3 Numerical derivative method2427.2.4 Application of gradient method by numerical derivative method 2467.2.5 Error back propagation method2517.2.6 CalculationE / .vkj 2527.2.7 CalculationE / .wji 2567.2.8 Realization of error back propagation method2627.2.9 Characteristics of learned neurons 2687.3 Using Keras to Implement Neural Network Model2707.3.1 Two layer feedforward neural network2717.3.2 Usage process of Keras 273Chapter 8 Application of Neural Network and Deep Learning (Handwritten Digit Recognition) 2778.1 MINST Dataset2788.2 Two layer feedforward neural network model2798.3 ReLU activation function2868.4 Space filter2918.5 Convolutional neural network2958.6 Pooling 3008.7 Dropout 3018.8 MNIST identification network model integrating various characteristics 302Chapter 9 Unsupervised Learning 3079.1 Two dimensional input data3089.2 K-means algorithm3109.2.1 Summary of K-means algorithm3109.2.2 Step 0: Prepare variables and initialize 3119.2.3 Step 1: Update R 3139.2.4 Step 2: Update μ 3159.2.5 Distortion measures 3189.3 Gaussian mixture model3209.3.1 Probability based clustering 3209.3.2 Gaussian mixture model3239.3.3 Summary of EM algorithm3289.3.4 Step 0: Prepare variables and initialize 3299.3.5 Step 1 (Step E): update γ 3309.3.6 Step 2 (Step M): update π, μ and ∑ 3329.3.7 Likelihood 336Chapter 10 Summary of the Book 339Postscript 349
Introduction: This book is the crystallization of years of work experience of senior network security teachers.The book systematically and deeply explains the combination of Python application examples and network security. It not only tells the practical application methods of Python, but also analyzes the network security programming technology of Python from the perspective of network security principles, and truly combines theory with practice.The book is divided into 16 chapters.Chapter 1 Overview1.1 Network security penetration test1.2 Carry out network security penetration test1.2.1 Communication with customers in the early stage1.2.2 Collecting love report1.2.3 Threat model1.2.4 Vulnerability Analysis1.2.5 Vulnerability utilization1.2.6 Post penetration attack1.2.7 Report1.3 Skills required for network security penetration test1.4 SummaryChapter 2 Basics of Kali Linux 22.1 Introduction2.2 Installing Kali Linux 2 ································· 102.2.1 Install Kali Linux in VMware virtual machine2.2.2 Install Kali Linux in Raspberry Pie 2 ··· 122.3 Common Operation of Kali Linux 22.3.1 File system2.3.2 Common commands2.3.3 Configure the network of Kali Linux 22.3.4 Installing third-party applications in Kali Linux 22.3.5 SSH remote control of Kali Linux 2 network2.3.6 Updating Kali Linux 22.4 Advanced Operation of VMware2.4.1 Installing other operating systems in VMware2.4.2 Network connection in VMware2.4.3 Snapshot and clone functions in VMware2.5 SummaryChapter 3 Python Language Basics3.1 Basics of Python3.2 Installing Python programming environment in Kali Linux 2 system3.3 Write the first Python program3.4 Selection structure3.5 Cycle structure3.6 Numbers and character strings3.7 List, tuple and dictionary3.7.1 List3.7.2 tuple3.7.3 Words3.8 Functions and modules3.9 Document processing3.10 SummaryChapter 4 Common Modules of Security Penetration Test4.1 Socket module documents4.1.1 Brief introduction4.1.2 Basic method4.2 Python nmap module file4.2.1 Brief introduction4.2.2 Basic method4.3 Scapy module documents4.3.1 Brief introduction4.3.2 Basic method4.4 SummaryChapter 5 Information Collection5.1 Basis of information collection5.2 Host status scanning5.2.1 Active host discovery technology based on ARP5.2.2 ICMP based active host discovery technology5.2.3 TCP based active host discovery technology5.2.4 UDP based active host discovery technology5.3 Port scanning5.3.1 Port scanning technology based on TCP full open5.3.2 TCP half open port scanning technology5.4 Service scanning5.5 Operating system scanning5.6 SummaryChapter 6 Infiltration of Vulnerabilities (Basic Part)6.1 Overflow and leak of test software6.2 Calculate the offset address of software overflow ······························· 1146.3 Look up JMP ESP instructions6.4 Preparation of penetration procedure6.5 Determination of bad characters6.6 Using Metasploit to generate shellcode ············· 1266.7 SummaryChapter 7 Infiltration of Vulnerabilities (Advanced Part)7.1 Brief introduction to SEH overflow7.2 Key points for writing SEH based overflow penetration module7.2.1 Calculate the offset to catch position ···· 1357.2.2 Find the POP/POP/RET address ··················· 1417.3 Preparation of penetration module 1427.4 Summary[2] Chapter 8 Network Sniffing and Deception8.1 Network data sniffing8.1.1 Write a network sniffer tool8.1.2 Calling Wireshark to view data package8.2 Principles and shortcomings of ARP8.3 Rationale of ARP deception8.4 Intermediary deception8.5 SummaryChapter 9 Attack of Denial of Service9.1 Data link layer denial of service attacks9.2 Denial of Service Attacks at the Network Layer9.3 Denial of Service Attack at the Transport Layer9.4 Application layer based denial of service attacks9.5 SummaryChapter 10 Identity Authentication Attacks10.1 Attacks on simple network service authentication10.2 Writing password cracking dictionary10.3 FTP violence cracking module10.4 SSH brute force cracking module10.5 Web violence cracking module ··························· 19410.6 Attacks on network authentication services using BurpSuite10.6.1 Forms based brute force cracking10.6.2 Bypass verification code (client) ················· 21210.6.3 Bypass verification code (server side) ·········· 21410.7 SummaryChapter 11 Programming Remote Control Tools11.1 Brief Introduction to Remote Control Tools11.2 Server side and customer side of remote control program11.2.1 Execute system command (subprocess module)11.2.2 Remote control server and client (socket module implementation)11.3 Convert Python script to exe file11.4 SummaryChapter 12 Wireless Network Penetration (Basic Part)12.1 Wireless Network Foundation12.2 Wireless function in Kali Linux 2 ············ 22912.2.1 Hardware requirements and software settings of wireless network sniffer12.2.2 Library documents for wireless network penetration12.3 AP scanner12.4 Wireless network data sniffer12.5 Client scanner of wireless network12.6 Scan hidden SSIDs12.7 Bypass target MAC filtering mechanism12.8 Capture encrypted data packet12.8.1 Capture WEP data package12.8.2 Capture WPA type data package12.9 SummaryChapter 13 Wireless Network Penetration (Advanced Part)13.1 Simulate the connection process of wireless client ······· 24113.2 Simulate the connection behavior of AP13.3 Preparation of Deauth Attack Program13.4 Wireless network intrusion detection13.5 SummaryChapter 14 Penetration Testing of Web Applications14.1 Module required for penetration test14.1.1 Use of requests library14.1.2 Other common module files14.2 Processing HTTP headers14.3 Cookies Handling14.4 Capture HTTP basic authentication data package14.5 Writing Web Server Scanner14.6 Scan all pages on the target server by force14.7 Security of verification code14.8 SummaryChapter 15 Generating Penetration Test Report15.1 Relevant theories of penetration test report15.1.1 Purpose15.1.2 Summary of contents15.1.3 Included scope15.1.4 Safely deliver the penetration test report ····· 26915.1.5 Contents of penetration test report15.2 Processing XML files15.3 Generate penetration report in Excel format ·········· 27115.4 SummaryChapter 16 Python Forensics Related Modules ········ 27916.1 Calculation of MD5 value16.1.1 Relevant knowledge of MD516.1.2 Calculate MD5 ············· 280 in Python16.1.3 Calculate MD5 ·································· 28016.2 Geolocation of IP address16.3 Time evidence16.4 Registry evidence16.5 Image evidence16.6 Summary
Points: 1Type: Technical documentUploader:HyperplatinumUpload time: May 30, 2021
Introduction: This book was written by Wes McKinney, the founder of Pandas project, and details the specific details and basic points of using Python to operate, process, clean and regularize data.Version 2 is a comprehensive revision and update of Python 3.6, covering the new version of pandas, NumPy, IPython and Jupyter, and adding a large number of actual cases, which can help you solve a series of data analysis problems efficiently.Major updates in Version 2 include:• All codes, including the update of Python tutorial to Python 3.6 (Python 2.7 is used in the first version)• Updated the installation guidelines for Python third-party release Anaconda and other required Python packages• Update the Pandas library to the new version in 2017• A new chapter on more advanced pandas tools and some tips• Brief introduction to the use of new statsmodes and scikit-learn
Introduction: This book combines machine learning, data analysis and Python language, and explains how to apply algorithms to practical tasks in an easy to understand way through cases.The book consists of 20 chapters, roughly divided into four parts.Part 1 introduces the necessary toolkit for Python, including Numpy, Pandas, and Matplotlib;The second part explains the classical algorithms in machine learning, such as regression algorithm, decision tree, integration algorithm, support vector machine, clustering algorithm, etc;The third part introduces the common algorithms in deep learning, including neural grid, convolutional neural network, recursive neural network;The fourth part is the project practice. Based on the real data set, the algorithm model is applied to the actual business.This book is suitable for beginners and enthusiasts interested in artificial intelligence, machine learning, data analysis and other directions.SummaryprefaceChapter 1 Guide to Artificial Intelligence1.1 Python is the first choice in AI era1.2 The core of artificial intelligence - machine learning1.3 Environment ConfigurationSummary of this chapterChapter 2 Numpy2.1 Basic Operation of Numpy2.2 Index and slicing2.3 Data type and numerical calculation2.4 Common functional modulesSummary of this chapterChapter 3 Data Analysis and Processing Library (Pandas)3.1 Data pre-processing3.2 Data analysis3.3 Common Function Operations3.4 Big data processing skillsSummary of this chapterChapter 4 Data Visualization Library (Matplotlib)4.1 Conventional drawing method4.2 Drawing of common chartsSummary of this chapterChapter 5 Regression Algorithm5.1 Linear regression algorithm5.2 Gradient descent algorithm5.3 Logical regression algorithmSummary of this chapterChapter 6 Logic Regression Project Practice - Credit Card Fraud Detection6.1 Data analysis and pre-processing6.2 Lower sampling scheme6.3 Logical regression model6.4 Oversampling schemeProject summaryChapter 7 Decision Tree7.1 Decision tree principle7.2 Decision tree pruning strategySummary of this chapterChapter 8 Integrated Algorithms8.1 Bagging algorithm8.2 Boosting algorithm8.3 Stacking modelSummary of this chapterChapter 9 Random Forest Project Practice - Temperature Forecast9.1 Random Forest Modeling9.2 Analysis of the impact of data and characteristics on the results9.3 Model parameter adjustmentProject summaryChapter 10 Characteristic Engineering10.1 Numerical characteristics10.2 Text characteristics10.3 Thesis and benchmarkSummary of this chapterChapter 11 Bayesian algorithm project practice - news classification11.1 Bayesian algorithm11.2 News classification taskProject summaryChapter 12 Support Vector Machine12.1 Operating principle of support vector machine12.2 Role of Support Vector12.3 Parameters involved in support vector machine12.4 Case: Influence of parameters on resultsSummary of this chapterChapter 13 Recommendation System13.1 Application of recommendation system13.2 Collaborative filtering algorithm13.3 Implicit semantic modelSummary of this chapterChapter 14 Project Practice of Recommendation System - Creating Music Recommendation System14.1 Data set cleaning14.2 Recommendation based on similarity14.3 Recommendation based on matrix decompositionProject summaryChapter 15 Dimension Reduction Algorithm15.1 Linear discriminant analysis15.2 Principal component analysisSummary of this chapterChapter 16 Clustering Algorithm16.1 K-means algorithm16.2 DBSCAN clustering algorithm16.3 Clustering ExamplesSummary of this chapterChapter 17 Neural Network17.1 Necessary foundation of neural network17.2 Overall architecture of neural network17.3 Network tuning detailsSummary of this chapterChapter 18 TensorFlow Practice18.1 Basic operations of TensorFlow18.2 Building Neural Network for Handwritten Font RecognitionSummary of this chapterChapter 19 Convolution Neural Network19.1 Principle of convolution operation19.2 Classic network architecture19.3 TensorFlow real combat convolutional neural networkSummary of this chapterChapter 20 Actual combat of neural network project - emotional analysis of film reviews20.1 Recurrent neural network20.2 Film review data feature engineering20.3 Building RNN modelProject summary
Points: 1Type: Technical documentUploader:HyperplatinumUpload time: May 30, 2021
Introduction: This book helps those who want to solve practical problems with mathematical tools. The only requirement may be to understand a little probability knowledge and programming.Bayesian method is a common mathematical method that uses probabilistic knowledge to solve uncertain problems. For a computer professional, he should be familiar with its application in such common computer problems as machine translation, speech recognition, spam detection and so on.However, this book will actually expand your horizon. Even if you are not a computer professional, you can also see the power of Bayesian methods in the war environment (the problem of German tanks in World War II), legal issues (the hypothesis test of kidney tumors), and sports gambling field (the problem of NFL competition between the Brown Bear team and the Canadian team).How can you judge the size of the German armored force from limited information, and how likely the team you support is to win the championship? In Dungeons and Dragons, what should you expect from the maximum value of the game character attributes? Even in ordinary paintball shooting games, having some Bayesian thinking can help you improve your game level.In addition, this book discusses how to solve more than ten practical problems in real life in 15 chapters.In the process of solving these problems, the author also imperceptibly helped the readers to form the methodology of modeling decision-making, how to choose between modeling error and numerical error, how to establish mathematical models for specific problems, how to grasp the main contradictions in the problem (key parameters in the model), and how to further optimize or verify the effectiveness or limitations of the model.In this sense, this book is also a successful sample of mathematical modeling.
Introduction: This book starts from zero basics, contains rich project case practice drills, and explains Python's environment construction, basic syntax structure, interesting applications, analysis and modeling, and complete project cases in detail.Chapter 1 Building the Python Development Environment1.1 Sharp tool 1: Notepad editor1.2 Sharp weapon 2: Anaconda1.3 Sharp weapon 3: Miniconda1.4 Sharp tool 4: PyCharm IDE tool1.5 Sharp tool 5: Spyder1.6 Sharp tool 6: Jupyter Notebook1.7 SummaryChapter 2 Python Data Type Usage2.1 Variables2.2 String2.3 List2.3.1 Add (append, insert, extend)2.3.2 Delete (pop, remove, del)2.3.3 Modification and check2.3.4 Loop traversal of list2.3.5 Sort, reverse2.3.6 Other operators of the list2.4 Set2.4.1 Creating Sets2.4.2 Add/Delete Sets2.4.3 Operations such as handover, merging and supplement of sets2.5 Dictionary2.5.1 Dictionary Search2.5.2 Adding and modifying dictionary2.5.3 Deleting a Dictionary2.5.4 Common methods of dictionary2.5.5 Ordered dictionary2.6 Functions2.7 SummaryChapter 3 Practical Application under Python3.1 Python connection to MySQL database3.2 Python connection to MongoDB database3.3 Stuttering segmentation and word cloud map3.4 Simple social network3.5 JSON parsing3.6 OCR character recognition3．7pyecharts3.8 Simple statistical analysis of stats3.9 SummaryChapter 4 Identification of Abnormal Samples4.1 Logical regression, cross validation and under sampling4.2 Identification of abnormal samples based on distribution4.3 SummaryChapter 5 Natural Language Processing Case - E-commerce Review5.1 Data loading and pre-processing5.2 Data visualization5.3 Text analysis5.4 Emotional analysis5.5 Text classification5.6 SummaryChapter 6 Model Fusion6.1 Fusion method of classification model6.2 Fusion method of regression model6.3 SummaryChapter 7 Create Financial Application Scorecard7.1 Variable selection7.2 Each variable is divided according to ln (odds)7.3 Calculating WOE and IV values7.4 Logical regression modeling7.5 Creating a scorecard7.6 Evaluation, use and monitoring of application score cards7.7 SummaryChapter 8 Social Network Analysis and Anti fraud8.1 Download and installation of Neo4j8.2 Introduction to graphical interface8.3 Cypher language8.4 Neo4j Case 1 - Analysis of the Character Relationship in Tian Long Ba Bu8.5 Neo4j Case 2 - Social Network Analysis in Financial Scenarios8．6Py2neo8.7 SummaryreferenceAppendix APyCharm Installation StepsAppendix BMySQL Installation StepsAppendix CMongoDB Installation StepsAppendix DNeo4j Installation ProceduresAppendix Ejdk Installation ProceduresAppendix F Installation Steps of Third Party Package
Introduction: Data science is a booming and promising industry. Some people call data scientists "the number one sexy career in the 21st century".This book explains data science work from scratch, teaches hacker skills necessary for data science work, and leads readers to become familiar with the core knowledge of data science - mathematics and statistics.The author chose a powerful and easy to learn Python language environment, built tools and implementation algorithms by hand, and carefully selected well annotated, concise and readable implementation examples.All codes and data covered in the book can be downloaded on GitHub.By reading this book, you can:Learn a Python crash course;Learn the basic methods of linear algebra, statistics and probability theory, and understand how they are applied in data science;Master how to collect, explore, clean, transform and operate data;Deeply understand the basis of machine learning;Various data models such as k-nearest neighbor, naive Bayes, linear regression and logical regression, decision tree, neural network and clustering are used;Explore recommendation systems, natural language processing, network analysis, MapReduce, and databases.Preface xiiiChapter 1 Introduction 11.1 Power of data 11.2 What is Data Science11.3 Incentive hypothesis: DataSciencester 21.3.1 Finding Key Contacts 31.3.2 Data scientists you may know 51.3.3 Wages and working years 81.3.4 Payment account 101.3.5 Topics of interest 111.4 Outlook 12Chapter 2 Python Express 132.1 Basic content132.1.1 Python acquisition 132.1.2 Zen of Python 142.1.3 Blank Form142.1.4 Module152.1.5 Algorithm162.1.6 Function 162.1.7 String172.1.8 Abnormality182.1.9 List 182.1.10 tuples 192.1.11 Dictionary202.1.12 Collection222.1.13 Control flow 232.1.14 True and False 242.2 Advanced content252.2.1 Ordering 252.2.2 List parsing 252.2.3 Generators and iterators 262.2.4 Randomness 272.2.5 Regular Expression282.2.6 Object oriented programming282.2.7 Functional tools292.2.8 Enumeration 312.2.9 Compression and Parameter Splitting 312.2.10 args and kwargs 322.2.11 Welcome to DataSciencester 332.3 Extended learning 33Chapter 3 Visualization Data343.1 matplotlib 343.2 Bar Chart363.3 Line diagram403.4 Scatter Chart413.5 Extended learning 44Chapter 4 Linear Algebra 454.1 Vectors454.2 Matrix 494.3 Extended learning 51Chapter 5 Statistics 535.1 Description of Single Dataset535.1.1 Center Inclination 555.1.2 Dispersion565.2 Relevant 585.3 Simpson Paradox 605.4 Other considerations for correlation coefficient 615.5 Correlation and Causality 625.6 Extended learning 63Chapter 6 Probability 646.1 Independency and independence 646.2 Conditional Probability656.3 Bayesian Theorem 666.4 Random Variables 686.5 Continuous distribution 686.6 Normal distribution 696.7 Central limit theorem 726.8 Extended learning 74Chapter 7 Assumptions and Inferences 757.1 Statistical hypothesis test757.2 Case: Coin toss 757.3 Confidence Intervals797.4 P-hacking 807.5 Case: running A/B test817.6 Bayesian inference 827.7 Extended learning 85Chapter 8 Gradient Descent 868.1 The idea of gradient descent 868.2 Estimated gradient 878.3 Using gradients 908.4 Selecting the correct step908.5 Comprehensive 918.6 Random gradient descent method 928.7 Extended learning 93Chapter 9 Obtaining Data 949.1 stdin and stdout 949.2 Reading a File969.2.1 Basis of text document969.2.2 Restricted documents979.3 Network Grabbing 999.3.1 HTML and parsing method999.3.2 Case: O 'Reilly Books on Data 1019.4 Using API 1059.4.1 JSON (and XML) 1059.4.2 Using API without verification 1069.4.3 Finding API 1079.5 Case: Using Twitter API 1089.6 Extended learning 111Chapter 10 Data Work11210.1 Exploring Your Data11210.1.1 Exploring one-dimensional data11210.1.2 Two dimensional data11410.1.3 Multidimensional data11610.2 Cleaning and revision11710.3 Data processing11910.4 Data Adjustment12210.5 Dimension reduction 12310.6 Extended learning 129Chapter 11 Machine Learning 13011.1 Modeling13011.2 What is machine learning 13111.3 Over fitting and under fitting 13111.4 Correctness 13411.5 Bias variance tradeoffs 13611.6 Feature extraction and selection13711.7 Extended learning 138Chapter 12 k Nearest Neighbor Method 13912.1 Model13912.2 Case: favorite programming language14112.3 Dimension Disasters 14612.4 Extended learning151Chapter 13 Naive Bayesian Algorithm15213.1 A simple spam filter15213.2 A complex spam filter15313.3 Implementation of algorithm15413.4 Test model15613.5 Extended learning 158Chapter 14 Simple Linear Regression 15914.1 Model15914.2 Using the gradient descent method16214.3 Maximum likelihood estimation16214.4 Extended learning 163Chapter 15 Multiple Regression Analysis 16415.1 Model16415.2 Further assumptions of the least squares model16515.3 Fitting model16615.4 Interpretation model16715.5 Goodness of Fit16715.6 Extraneous remarks: Bootstrap 16815.7 Standard error of regression coefficient 16915.8 Regularization17015.9 Extended learning 172Chapter 16 Logical Regression 17316.1 Question 17316.2 Logistic function17616.3 Application model17816.4 Goodness of Fit17916.5 Support Vector Machine18016.6 Extended learning184Chapter 17 Decision Tree 18517.1 What is a decision tree 18517.2 Entropy 18717.3 Entropy of segmentation 18917.4 Creating a Decision Tree 19017.5 Comprehensive application19217.6 Random forest 19417.7 Extended learning195Chapter 18 Neural network19618.1 Percept19618.2 Feedforward neural network19818.3 Back propagation 20118.4 Example: Defeat CAPTCHA 20218.5 Extended learning206Chapter 19 Cluster Analysis 20819.1 Principle 20819.2 Model20919.3 Example: Party 21019.4 Select the number of clusters k 21319.5 Example: clustering colors 21419.6 Bottom up hierarchical clustering 21619.7 Extended learning221Chapter 20 Natural Language Process22220.1 Word cloud 22220.2 n-grams model22420.3 Grammar22720.4 Extraneous remarks: Gibbs sampling 22920.5 Theme modeling23120.6 Extended learning236Chapter 21 Network Analysis23721.1 Intermediary centrality 23721.2 Eigenvector centrality 24221.2.1 Matrix Multiplication24221.2.2 Centricity 24421.3 Directed graph and PageRank 24621.4 Extended learning248Chapter 22 Recommendation System24922.1 Manual screening 25022.2 Recommended popular food 25022.3 User based collaborative filtering method25122.4 Item based collaborative filtering algorithm25422.5 Extended learning256Chapter 23 Database and SQL 25723.1 CREATE TABLE AND INSERT 25723.2 UPDATE 25923.3 DELETE 26023.4 SELECT 26023.5 GROUP BY 26223.6 ORDER BY 26423.7 JOIN 26423.8 Sub query 26723.9 Indexing 26723.10 Query optimization 26823.11 NoSQL 26823.12 Extended learning269Chapter 24 MapReduce 27024.1 Case: Word Count27024.2 Why MapReduce 27224.3 More general MapReduce 27224.4 Case: analysis status update27324.5 Case: matrix calculation27524.6 Extraneous remarks: combiner27624.7 Extended learning 277Chapter 25 Data Science Foresight 27825.1 IPython 27825.2 Mathematic27925.3 Don't start from scratch 27925.3.1 NumPy 27925.3.2 pandas 28025.3.3 scikit-learn 28025.3.4 Visualization28025.3.5 R 28125.4 Searching for Data28125.5 Engaging in Data Science28125.5.1 Hacker News 28225.5.2 Fire fighting vehicles28225.5.3 T-shirt 28225.5.4 What about you? 283About the Author 284About the cover 284
Points: 1Type: Technical documentUploader:HyperplatinumUpload time: May 30, 2021
Introduction: NLTK is a very popular and widely used Python library in the field of natural language processing.The advantage of NLTK lies in its simplicity. Most of the complex natural language processing tasks can be completed with a few lines of code.This book aims to describe how to use Python and NLTK to solve various natural language processing tasks and develop machine learning applications.This book introduces the basic modules of NLTK, describes a lot of techniques for implementing natural language processing with NLTK, discusses some text processing methods and language processing technologies, and shows a lot of practical experience in implementing NLP projects with Python.This book mainly covers all the preprocessing steps required in the text mining/NLP task, how to use NLTK 3 of Python 3 for text processing, and how to carry out NLP projects through Python.This book is suitable for NLP and machine learning enthusiasts, Python programmers and researchers in the field of machine learning.
