A strong correlation has been discovered between items such as water bottles, batteries and flashlights AND hurricanes. Every time there is uptick in the the sale of water bottles, batteries and flashlights, it surely signals that hurricane is on its way in next 24 to 72 hours. So far, this has been proved with data time and time again.
Please take a look at the following:
| SN | Area | WB | FL | Batt | TTHA |
| 1 | Texas | 61% | 92% | 65%…
In ML Use Case — Part 1, we established the need for machine learning example, with examples. In this part let us see how to implement the use case on Databricks platform.
What is the use case:
Those who know what…
In this age of hyper competition, machine learning use cases hold the key to success in the market. Oh well, you must have heard that statement multiple times, hyper competition, super regional, micro focused etc. I don’t even know and remember the various combinations ‘these’ words any mover. However there is a good amount of truth in it.
Let us take few example that one encounters on regular basis and have some discussion around it.
One can’t make a larger shirt with less material. Additional material should increase the price. Apparently, it is not so with Dockers shirt as shown…
There are lot of definitions of game theory, such as mathematics of conflict of interest. Decision making is about single agent taking decisions, however in real world, there is almost always other agent or agents who are taking rational, self-interested decisions along with you. Hence game theory is also about multiple agents making optimal decisions. It was and is used in lot of practical subjects such as economics, politics and biology. Game theory is increasing becoming part of AI/ML.
Minimax ≡ Maximin
Consider a game being played by two agents, where both the agents are trying to maximize their value…
RL can be thought about as an API. The approach of taking a model with transitions and rewards and converting it into policy is called as planning. The code is called as planner. Similarly transitions can be converted to policy using ‘Learner’ code, and is called and reinforcement learning process. There are sub components as well — modeler, which takes in transitions and convert to model; and simulator which takes model and convert it into transitions. Note that these ‘API’s can be linked to create different kind of combinations as follows:
There are three main types of learning as follows:
1. Supervised Learning — where label for the data is available
2. Unsupervised Learning — where the label for the data is not available
3. Reinforcement Learning — where agents learns actions in order to maximize ‘cumulative’ rewards.
RL focuses on finding balance between exploration of uncharted data, and exploitation of current knowledge of the data.
The environment (or the world) is typically stated in the form of Markov Decision Process (MDP)
Markov Decision Process
MDP is a framework, which consists of following elements:
States : S
Model : T(s, a, s’)…
In machine learning context, every input vector X and output vector Y can be considered as probability density function. Information Theory is mathematical framework which enables us to compare these probability density functions to ask questions such as — are these input vectors similar? does this feature has any information at all?
Entropy — is a measure of unpredictability of the state or average information content. So if the coin is fair, i.e. probability of head or tell is exactly half, then then entropy of the event of tossing a coin is 1. Inversely, if the coin is unfair, and…
For discovering the knowledge residing within features, understanding interpret-ability, and gathering insights about the data, feature selection is important. The data size required for data selection grows exponentially, that is for n features, we need 2^N data. The time complexity for this problem is exponential.
In this feature selection technique, the features are selected first and them fewer features are passing to learning algorithm. The outcome of the learning algorithm is inspected manually (or using program) and different set of features are selected. Filter is faster than wrapping, as filter (or search) does not worry about the features required…