An example


						# Idealized Representation of Embedding
						books = ["War and Peace", "Anna Karenina", 
								  "The Hitchhiker's Guide to the Galaxy"]
						books_encoded_ideal = [[0.53,  0.85],
											              [0.60,  0.80],
											              [-0.78, -0.62]]
					    ...

An example


						# Idealized Representation of Embedding
						books = ["War and Peace", "Anna Karenina", 
								  "The Hitchhiker's Guide to the Galaxy"]
						books_encoded_ideal = [[0.53,  0.85],
											              [0.60,  0.80],
											              [-0.78, -0.62]]
						Similarity (dot product) between First and Second = 0.99
						Similarity (dot product) between Second and Third = -0.94
						Similarity (dot product) between First and Third = -0.97

(hint)dot product: x1y1 + x2y2

Answer

Cosine Answer : C-A-B

Dot Answer(Norm) : A-B-C

Eculidean Distance : C-B-A

Based on the feature matrix above and using dot product `$$ \sum_{n=1}^{d} x_i y_i $$`let user embedding as $x$ while app embedding as $y$ (binary vectors) - if the feature both appearing in $x$ and $y$, then `1 * 1 = 1`.

#### The Good and the Bad | The Good | The Bad | | --- | --- | | Easily scalable | Recommendations limited to users' existing interests | | Can capture specific interests of users | Requires a lot of domain knowledge for feature engineering |

## Short Quiz 3 Calculate the dot product for each app in the preceding app problem. Which app should we recommend? - A. The casual app created by TimeWastr. - B. The educational app created by Science R Us. - C. The health app created by Healthcare.

## answer B. The educational app created by Science R Us.

Collaborative Filtering

Collaborative filtering uses similarities between users and items simultaneously to provide recommendations. Hence collaborative filtering can recommend an item to user A based on the interests of similar user B.

From the figure above, notice that there are two types of collaborative filtering: user-based and item-based. Let's look at user-based first.

#### The Good and the Bad | The Good | The Bad | | --- | --- | | No domain knowledge necessary | Cannot handle fresh items during production | | Can help users discover new interests | Hard to include side features |

### Recommendations using Deep Neural Networks Collaborative filtering learn embeddings via matrix factorization. This method is limited in terms of: 1. Hard to include side features 2. Popular items tend to be recommended for everyone

**DNN models can address these limitations of matrix factorization** > easily incorporate query features and item features (due to the flexibility of the input layer of the network), which can help capture the specific interests of a user and improve the relevance of recommendations.

For the sake of time, we will not touch DNNs further. You can find out further [here](https://developers.google.com/machine-learning/recommendation/dnn/softmax).

### Retrieval After candidate generation (means that we have an embedding model), the system can do two (2) things at serve time:

1. Matrix factorization - query embedding is known statically: Simply look up query embedding from embedding matrix 2. DNN model - query embedding unknown: Simply compute query embedding at serve time (inference)

Now that we have: 1. Query embedding, $q$ - represented by blue circle below; 2. Item embeddings, $V_j$ - represented by green circle below;

then we can return top K items using similarity score $s(q, V_j)$. ![Retrieval](https://developers.google.com/machine-learning/recommendation/images/2Dretrieval.svg)

### Scaling Up To find nearest neighbours, the system can exhaustively score every potential candidate - but it can be very expensive for large corpora. Possible workarounds: 1. Statically-known query: Offline exhaustive scoring -> precompute and store list of top candidates 2. Unknown query: Approximate nearest neightbour method

## Scoring Generated candidates can come from multiple candidate generators using different sources: - Related items - User features - Geographic info - Popular or trending items - Social graph (liked or recommended by friends)

Querying the model with all possible positions is too expensive and the system still might not find a consistent ranking across multiple ranking scores: 1. Create position-independent rankings 2. Rank all candidates as if they are in the top position on the screen

## Re-ranking Re-ranking can improve the recommendations by considering additional criteria or constraints:

1. **Use filters to remove some candidates** - i.e. removing click-baits: - Training a separate model that detects whether a video is click-bait. - Running this model on the candidate list. - Removing the videos that the model classifies as click-bait.

2. **Transform the score returned by the ranker** - i.e. modifying score: - As a function of video age (promote fresher content) - As a function of video length (increase viewing time)

**So, What are the challenges?**

1. **Freshness**: Aim to incorporate the latest usage information, e.g. current user history and the newest items - Warm-start and re-run training as often as possible - Create an "average" user to represent new users in matrix factorization models - Use a DNN such as a softmax model or two-tower model - Add document age as a feature

2. **Diversity**: Lack of diversity can cause a bad or boring user experience - Train multiple candidate generators using different sources - Train multiple rankers using different objective functions - Re-rank items based on genre or other metadata

3. **Fairness**: Treat all users fairly and reduce unconscious bias in data - Include diverse perspectives in design and development - Train ML models on comprehensive data sets and add auxilliary data when certain groups are underrepresented - Track metrics on each demographic to watch for biases - Make separate models for underserved groups

## Conclusion

1. Recommendations are the suggestions served up based on other things the user (e.g. you) like and they allow users to get in touch with compelling content in large corpora, particularly items that the user might not have thought to search for their own.

2. Recommendations can be performed via three (3) steps: - Candidate generation - Scoring - Re-ranking

3. Items and queries can be represented using embeddings.

4. Candidate generation is performed via: - Content-based filtering - Collaborative filtering (user or item-based) - Deep neural networks

5. Scoring can be performed for: - Maximizing click rate, but might promote click-baits - Maximizing watch time, but might promote long videos - Both increase diversity and maximize watch time

6. Re-ranking can be performed via: - Filtering to remove candidates - i.e. click-baits - Transformt the score - i.e. fresher content 7. Challenges for a recommendation system: - Freshness of model - Diversity of content - Fairness for users

## Demo Please visit this [Streamlit app](https://share.streamlit.io/mnobeidat13/handm-recommender-system/main) Streamlit app for the recommendation system demo.
![Retrieval](./assets/qrcode_streamlit.png)

THE END

- Try to read our document
- Google Crash Course