# Introduction
Vector AI's Guide To Vectors
Our aim is to make vectors accessible and to ensure that using vectors in applications is a magical experience. We believe vectors will change how the world builds applications and processes and want to help as many individuals and businesses do this.
This book has been written by the Vector AI team and you can find out more about what we do [here](https://getvectorai.com/). Along with this book, we also have other available resources and materials others can use and explore vectors such as our [hub](https://hub.getvectorai.com/), [playground](https://vectorai-playground.web.app/) and our open-source Github packages ([Vector AI](https://gh.vctr.ai) and [VectorHub](https://github.com/vector-ai/vectorhub)).
# Guide to this Book
How to use this Book
This book aims to introduce a range of audiences to vectors. From first learning about vectors, using vectors for building search experiences to more difficult topics such as optimizing vectors for personalized search engines and detecting biases in vectors.
This book is designed such that each piece of content exists in bite-sized chunks to allow for easier information consumption for the reader and intended to maximize utility and readability as much as possible.
**Intended Audiences**
Due to the varying levels at which different individuals can approach vectors, each article is written with pre-requisite knowledge and intended audiences. However, the intended audiences may not be exactly exhaustive and readers will often find that our articles are quite easy to digest despite the higher technical bar that has been set. As a result, even if the reader is not part of the intended audiences, they may still read on and continue to understand we encourage readers to also give feedback on where the reading can improve.
**Codes and Content**
The codes are designed in the following way:
| Course Code | Level |
| ----------- | ------------------------------------------------------------------------------------------- |
| 🌱 | Beginner (Usually suited for people new to a topic) |
| 🧍 | Intermediate (Usually suited for people looking to use the Vector AI playground) |
| 🧮 | Advanced (Mathematical) (Suited for people looking for a deeper mathematical understanding) |
| 💻 | Advanced (Code) (Suited for people looking to use the API or SDK) |
**Why write this book?**
Vector AI's mission is to accelerate the development of artificial intelligence products using vectors. We see vectors as a technology that will change the future so we want to make using vectors as easy as possible. We discovered that the general data scientist/public does not know much about vectors so we set out to write this book to help others learn and understand them for their use. Along the way, we include tips on how to use vectors and how they can be used with the Vector AI API.
# Introduction to vectors
An introduction to vectors
**Assumed Knowledge**: Vectors\
**Target Audience**: General Audience\
**Reading Time:** 3 minutes
**What are vectors?**
Vectors are a list of numbers that meaningfully and uniquely represent data. See below for an example.
```
[0.324, 0.241, 0.934, 0.424, 0.141, 0.242] #example of a vector
```
Although vectors may look like a series of random numbers, they are actually the result of carefully constructed and trained artificial neural networks (more details below).
**Think of vectors as the fingerprint of data**. Much like how everyone has their own fingerprint, every piece of data (whether it is an image, video, text or audio) has its own vector.
**How are vectors constructed?**
Vectors can be constructed from:\
1\) A row of data (for example - the machine learning model is an excellent example\
2\) Extracting a layer in the middle of a neural network
Neural networks allow us to provide a new way to see the work done.
# Applications of vectors
A discussion on the applications of vectors
**Assumed Knowledge**: Vectors\
**Target Audience**: General audience\
**Reading Time:** 2 minutes
There are 10,000+ applications of vectors, here are just a couple of examples:
By media type:
* **Natural language processing:** question-answering, semantic matching across languages, text search within and across PDFs, lyrics search
* **Image:** Reverse image search, product recommendations, outfit matching, facial identification
* **Audio**: Voice capture and generation
* **Video:** Video search, Frame by Frame search
* **Image-Text**: Search images using text, searching text using images
By industry:
* **Education**: Course matching, candidate-matching, Course FAQ, Course Question-Answering, Coach matching
* **Fraud**: Fraud determination through vectors
The list of applications goes on and on
# Vectors for classification
Vectors are re-framing how we are approaching traditional deep learning problems.
**Required Knowledge**: Vectors, Encoding, Classification problems\
**Audience**: Data scientists, Vector enthusiasts, Statisticians, Machine learning engineers\
**Reading time**: 5 minutes
**Defining Multi-Classification**
Classification refers to when a model is used to predict 2 or multiple labels (more than 2). As an example, this could be when given an animal image, the individual is required to label the category that the animal belongs to from a list of pre-defined categories.
**Traditional Classification**
In the classification example above, the image is read and fed through a neural network. The neural network, trained on predicting whether an image is a dog/cat/rabbit/emu is then given a probability that it can belong in each class.
**Vectors reframe traditional classification into a vector search problem**
Let us reframe our example of image classification (labeling an image based on the given captions to identify the best category) using vector search. Instead of predicting the most likely label using a neural network (which is how it was previously done), the labels were, instead, encoded using a deep learning model. The images were also encoded and then a vector search was performed on the projections of these encodings to identify the most similar images to the labels.
Let us now quickly compare the advantages and disadvantages of each approach.
**Advantages and Disadvantages of Vector Similarity Approach**
There are several advantages to this approach:
* Resolves the cold-start issue (in traditional approaches, classification neural networks would have to be re-trained in order to adapt to new categories)
* Reduced cost of data science experiments - using excellent out-of-the-box vectors/similarity search that resolves this issue means you can then reduce the cost of initial data science experiments and bring data to value quickly
Key Disadvantage:
* If you require vectors to fit well on pre-defined, it requires more data science expertise to finetune these vectors compared to traditional multi-classification approaches.
# Limitations of vectors
An outline of the limitations of what vectors are capable of.
**Assumed Knowledge**: Vectors\
**Target Audience**: General audience\
**Reading Time:** 2 minutes
Vectors are largely dependent on the models being used.
There are 2 main difficulties associated with using vectors in production engineering systems:
* The difficulty of obtaining the right type of vectors
* The difficulty of productionisation of vectors
Let us go over the difficulties of each.
### The Right Vectors
Choosing the right vectors for your use case can be difficult. When choosing a model for vectors, you need to ask yourself the following:
* Which out-of-the-box model works best for this use case?
* Is an out-of-the-box model good enough? If not, how do I fine-tune my vectors for my particular use case?
* What is the model trained on?
* If the model's vectors do not perform well - is it because it was not trained on the right data or was the training method not suitable?
#### Why do some models need to be fine-tuned and how does that help?
Firstly - what is finetuning? In the context of vectors, fine-tuning refers to the process of altering your vectors by changing the weights in your model. You can change the weights in your model by training the model further on your specific dataset so that similarities between certain images and items can be improved.
To better understand the importance of finetuning, let us consider an example scenario of how Image2Vec can fail in identifying the same faces and why we may need to finetune a new model to produce Face2Vec. Image2Vec has been trained to identify similar images and this can be based on color, the orientation of the image, features of the image. However, it may prioritise images of 2 different people if they are in the same orientation. As a result, to improve the similarity between the same person in spite of orientations - we can finetune the model the objective of having higher similarity between the same individual over the same orientation.
### Productionisation difficulty
Putting vectors into production can be a difficult task. One has to build out an API for the index, the encoder, provide options for advanced search functions with vectors and then figure out ways to then store search. There is a lot of work that has been done simply to get it running but more work needs to be done to get it to a useful state for productionisation.
# What is vector search?
An introduction to vector search/nearest neighbors.
**Assumed Knowledge**: Vectors\
**Target Audience**: General developers\
**Reading Time:** 3 minutes
**What is vector search?**
Vector search is the process of finding the most similar vectors to itself. If you are unfamiliar with vectors, I recommend reading about them [here](/what-are-vectors/untitled).
Although we initially used the analogy of vectors as fingerprints in our introduction, vectors actually have additional properties that allow them to be useful in practical applications. These include:
1. Similar data have similar vectors.
2. You can measure the similarity of these vectors statistically in a number of different ways.
The most common algorithms that are used are called **nearest neighbor algorithms**. You can read more about them [here](https://en.wikipedia.org/wiki/Nearest_neighbor_search).
**Why is vector search important?**
If you can find similar vectors based on the data - this means, you can provide different ways of linking data in ways individuals may have never considered. Linguistically - you can now link sentences based on semantics as opposed to relying on co-occurrences of words (used in traditional word search). Similarly, for image search, you can use reverse image search and personalised image search -- allowing for better recommendations for searches. If you are interested in vector search applications, you can read about them [here](/what-are-vectors/untitled/applications-of-vectors).
# How to vector search
A Guide On Using Vector Search
**Assumed Knowledge**: Vectors\
**Target Audience**: General audience\
**Reading Time:** 1 minute
Vector Search is reliant largely on index libraries and open-source models. Each vector search guide follows a template of:
The steps/materials can be summarised in the following way:\
**Data**: Obtain the data in a way that can be processed into a numerical representation and fed through the necessary model.\
**Encode**: Feed the data's numerical representation into a model and extract the vector which can be indexed and searched.\
**Index**: Indexing the vectors (from which the data has been encoded) in an efficient way that allows for retrieval.\
**Search**: Search the vectors that have been indexed by using a variety of Nearest Neighbor algorithms, filters, chunking and queries.
**The Difficulties of Vector Search**\
While the above process appears simple, there are a lot of difficulties with actually using vector search for production. These difficulties include:
* Deploying your index and search for production
* Usage of vectors to optimise search results
* Optimising the way search is being done on the vectors
* Optimising the matching of user intent and products
# How to build image to text search using code
A guide on building text to image/image to text search with Vector AI
**Assumed Knowledge**: Vectors, Vector Search, Python (Basic level)\
**Target Audience**: General Developer, Data Scientist, Python Developer\
**Reading Time:** 5 minutes\
**Requirements**: Python 3.6 or Python 3.7
To build text to image search, you will need a text to image model. An example of a text to image model is OpenAI's CLIP. You can read more about CLIP as a model [here](https://hub.getvectorai.com/model/text_image%2Fclip).
```
!pip install vectorhub[clip]
from vectorhub.bi_encoders.text_image.torch import Clip2Vec
model = Clip2Vec() # This will download and run the model
```
To encode the text, you can run the following in Python:
```
model.encode_text("This is a dog.")
```
To encode images, you can run the following in Python (if you are curious about the encoding process for images, you can read about it [here](/vector-ai-documentation/how-to-turn-data-into-vectors-code/how-to-turn-images-into-vectors).
```
model.encode_image("https://cdn.britannica.com/88/154388-050-11BCAE3C/CEO-Elon-Musk-SpaceX-car.jpg")
```
Now that the model has been instantiated, we simply need a way to store our index and then retrieve the relevant image or text. For the rest of the guide, we will use the Vector AI client but if you are interested in using a completely open-source indexing tool, we recommend our other article on using [FAISS](/vector-ai-documentation/semantic-nlp-search-with-faiss-and-vectorhub).
To add vectors to Vector AI's cloud index, simply run the following:
```
!pip install vectorai
from vectorai import ViClient
vi = ViClient(username, api_key)
```
From here, we want to insert documents into the Vector AI database. Vector AI uses **document-based storage**. Document-based storage allows Vector AI to be more useful for storing metadata and you can read more about it [here](https://gh.vctr.ai).
```
# Create your list of data
docs= [
{'image_url':'https://cdn.britannica.com/88/154388-050-11BCAE3C/CEO-Elon-Musk-SpaceX-car.jpg',
'label': 'Elon musk'},
{'image_url':'http://cdn.cnn.com/cnnnext/dam/assets/180316113418-travel-with-a-dog-3.jpg',
'label': 'dog'},
{'image_url':'https://cdn.mos.cms.futurecdn.net/vEcELHdn998wRTcCzqV5m9.jpg',
'label': 'laptop'}
]
collection_name = 'sample_image_text'
vi.insert_documents(collection_name, docs, models={'image_url': model.encode_image})
```
Now that you have inserted documents, you will now want to search your collection. This can be done using the following:
```
query = "Good boy"
text_query_vector= model.encode(query)
vi.search(collection_name, vector=text_query_vector, vector_field='image_url_vector_')
```
Some users may be confused by the name of the vector field. Where does`image_urlvector_` come from? Vector AI automatically encodes vectors and adds a `_vector_`to the encoded field. If you are ever confused by what the vector names are, simply refer to the collection schema of the collection to understand what is happening.
```
vi.collection_schema(collection_name)
# Returns a dictionary of the fields in the collection and their respective values.
```
If you want to use this in your own applications that are not Python-based, you can then search this using the Vector AI API, which has been documented [here](https://api.vctr.ai/documentation).
# Try vector search with playground!
Vector AI's playground makes experimenting with vector search easy!
**Assumed Knowledge**: Vectors\
**Target Audience**: General audience\
**Reading Time:** 3 minutes
Vector AI's playground makes experimenting with vector search easy!
You can try out our playground here:
In the playground, you will see the different applications vectors can provide and all the different ways that vectors can be used (intelligent feeds, image to text search, personalised recommendations). You can test the effectiveness of the results for yourself! If you are interested in building your own, we recommend our other articles.
# Vector search with code
A guide to building vector search with Vector AI
**Assumed Knowledge**: Vectors, Vector Search, Python (Basic level)\
**Target Audience**: General developer, Data scientist, Python developer\
**Reading Time:** 5 minutes\
**Requirements**: Python 3.6 or Python 3.7
Below, we build a simple example of image search with Vector AI
1. We get the data on a document-based approach.
```
collection_name = 'pokemon_images'
documents = []
for i in range(1, 20):
documents.append({
'image': 'https://assets.pokemon.com/assets/cms2/img/pokedex/full/{}.png'.format(f'{i:03}'),
'pokemon_id' : str(i),
'_id': i
})
```
2\. We encode the images and instantiate the Vector AI client. If you do not have a username or API key, simply request one Pythonically from this link.
```
from vectorai.client import ViClient
vi_client = ViClient(username, api_key, url)
from vectorai.models.deployed import ViImage2Vec
image_encoder = ViImage2Vec(username, api_key, url)
for doc in documents:
doc['image_vector_'] = image_encoder.encode(doc['image'])
```
3\. Add your documents to your index using `insert_documents`.
```
vi_client.insert_documents(collection_name, documents)
```
4\. Search your documents using `search.`
```
# Search the application
image_url = 'https://assets.pokemon.com/assets/cms2/img/pokedex/full/003.png')
search_results = vi_client.search(
collection_name,
image_encoder.encode(image_url), search_fields=['image_vector_'],
page_size=5)
# Show the results!
vi_client.show_json(search_results, image_fields=['image'], image_width=150)
```
# Terminology Guide
Guide to the terminology used in Vector AI
An example of a document in Vector AI:
```
document_example = {
"car": {
"wheels":
{
"number": 4
}
}
}
}
```
| Terminology | Definition |
| ------------------------ | -------------------------------------------------------------------------------------------------- |
| Vectors | AKA embeddings, 1D arrays, latent space vectors |
| Models/Encoders | Turns data into vectors (e.g. Word2Vec turns words into vectors) |
| Vector Similarity Search | Nearest neighbor search, distance search |
| Collection | Index, Table (a collection is made up of multiple documents) |
| Documents | (AKA JSON, item, dictionary, row) - a document can contain vector and other important information. |
| Field | A field is the key to a Python dictionary. |
| Value | A value is the value of a Python dictionary |
# Inserting Into Vector AI
How to insert your data into Vector AI
Before inserting into Vector AI, there are a few important things to note about Vector AI.
1\) Document-based storage. Vector AI relies on a document-oriented storage. An example of a document can be seen below.
```
document_example = {
"car": {
"wheels":
{
"number": 4
}
}
}
```
2\) When uploading documents into VectorAI, the schema is automatically determined. The schema itself relies on the following built-in concepts:
| Field Name | Description |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `_id` | This refers to the unique identity of the document. Every document should have its own ID value. |
| contains `_vector_` | If labelled with `_vector_` then the value will be treated as a vector and will be treated as a variable. Note: This needs to be an array of floats. (Note: In Python, lists are arrays and we are not referring to NumPy arrays.) |
| contains `_chunkvector_` | A `_chunkvector_` refers to a vector that is part of a larger document. Documents can have multiple chunkvectors where each chunk is part of a larger document. |
# Inserting with playground
How to insert documents using PlayGround
Vector AI's playground provides users with a simple way to insert data via CSV. Users are able to upload their data, insert them into Vector AI and then encode them according to the type of model they want.
{% embed url="" %}
A tutorial on using the playground for inserting and encoding data
{% endembed %}
# Inserting with API
Inserting with the Vector AI API
If you are looking for more flexibility when you are inserting your documents via the API or Python SDK, then the following is for you:
When inserting your data into Vector AI, you will need a way to encode vectors as you insert. As a result, we have built a few ways to allow users to flexibly encode vectors.
There is no right endpoint for inserting. Different users will have different insertion and encoding preferences based on technical requirements. A few examples of such situations:
* Shirley (data scientist) needs to insert her data into Vector AI. However, her model is not yet deployed and she needs to test the results before deploying to ensure her vectors work as intended. So she encodes all her documents prior to inserting and searches using her locally saved model.
* Krissy (data engineer) has just deployed her model. However, over time, she realizes she can't keep encoding her models locally as she is not always at her computer. Instead, she ploys her model and ensures that she is able to encode while inserting to ensure that her stakeholders are using the right data.
* Tom (machine learning engineer) has a serious amount of data. He realizes he needs to run an encoding job later and try out different vector searches. For this, he quickly inserts them all into Vector AI and then runs multiple different encoding jobs that adds new vectors afterwards.
# Inserting with API - encoding while inserting (recommended)
A guide to encoding while inserting
If you are looking to encode while inserting, then you will need a deployed model or rely on one of Vector AI's models. The main advantage of this is that you can immediately get your deployment to start working. You can take a look at the following for inserting and encoding.
```python
import requests
url = "https://vectorai-development-api.azurewebsites.net/collection/insert_and_encode"
requests.post(
url=url,
json={
"username": username,
"api_key": api_key,
"collection_name": collection_name,
"documents": documents,
"encoders": [
]
}
)
```
# Inserting with API - encoding before inserting
A simple tutorial on encoding before inserting
## Option 1 - Encoding Before Inserting
Encoding before inserting can often be the best decision when you have a locally saved model and want to test it out without having to deploy it. This allows you to quickly test if the model will be a good fit (and is even faster if you are using the Vector AI API and SDK).
If you are looking to insert, you will be using the following API endpoint.
{% tabs %}
{% tab title="Python" %}
```python
import requests
url = "https://vectorai-development-api.azurewebsites.net/collection/bulk_insert"
response = requests.post(
url=url,
json={
"username": "string",
"api_key": "string",
"collection_name": "string",
"documents": [{},{}],
"insert_date": true,
"overwrite": true,
"update_schema": true,
"quick": false,
"pipeline": [ ]
}
)
```
{% endtab %}
{% tab title="Javascript" %}
```
```
{% endtab %}
{% endtabs %}
The documentation for this endpoint can be found in the API.
If the collection name does not exist, Vector AI will automatically create a collection for you so you can just insert properly.
# Inserting with API - encoding after inserting
A guide to encoding after inserting into Vector AI
# How to check insertion succeeded
A guide to checking insertion worked successfully
# Searching with Vector AI
How to search with Vector AI
Try out search with Vector AI using the playground!
{% embed url="" %}
If you are interested in using the API endpoint instead, feel free to read other articles.
# How to search with the playground
How to try out vector search in the playground
The Vector AI playground makes it easy to try out vector search. This vector search experience automatically shows images/text/audio in easy-to-read formats with carefully chosen defaults to ensure a seamless search experience. From here, after testing, it is easy to give the best configuration to your developers so they are aware of how to use the right endpoint.
{% embed url="" %}
# Combining with traditional search
A comparison of vector search and traditional search.
**Assumed Knowledge**: Vectors\
**Target Audience**: General Audience\
**Reading Time:** 3 minutes
If you are interested in comparing vector and traditional search, we recommend firstly looking over the playground which can be found here: .
While the results may appear obvious, we will go over how this works and is built.
**Firstly - what is traditional search?**
Traditional search is reliant on words and the number of times a word may appear in a specific document. For example, if we are searching for the word "dog", sentences where the word "dog" appear more times will end up occurring. However, as a result, if we search for the question "what is a dog", we will note that no results will come up because there may be no results that contains all the words.
**So how does vector search resolve this?**
Vector search instead identifies the meaning behind the strung together text so instead of interpreting every word for what it is worth - it identifies the semantic meaning of the strung together text and outputs a vector for search on all the results. As a result - vector search becomes a useful tool.
**However - we can sometimes get the best results from combining traditional search with vector search**.
We can sometimes get better results from combining traditional search with vector search. This is useful because it takes into consideration the exact words we type a well as the meaning of the strung-together text. Vector AI allows us to do this using the hybrid search API. You can find out more about its use case and application in [Combining Vector Search With Traditional Search](/vector-search/searching-with-vector-ai/vector-search-vs-traditional-search/combining-vector-search-with-traditional-search).
# How to combine exact text search with vector search
A guide on how to add exact text search with vector search
Adding exact text search with vector search is easy and provides users with a number of ways to improve.
{% embed url="" %}
# How to add exact text search to vector search
Introducing hybrid search and its importance
**Assumed Knowledge**: Vectors\
**Target Audience:** General Audience / Python developers\
**Reading Time:** 3 minutes
There are sometimes search cases where pure vector search often does not provide the best solution. For example - we can take the simple case of product SKUs in retail e-commerce. Product SKUs are the name of the products.
### E-Commerce Case Study
**Weakness Of Vector Search**
Below, we use an example of vector search where an individual searches for an SKU. However, the search results encode the letters and fail to realize/return the right SKU. We also attach a code example using the Vector AI client for those interested in trying this out.
```
search_results = vi_client.search(
collection_name,
text_encoder.encode('R170NZKAXSA'), 'name_vector_', page_size=3)
vi_client.show_json(search_results, selected_fields=['_id', 'name', 'sku'],
image_fields=['image_url'], image_width=150)
```
In these situations, our search should properly return the right value when given the SKU. In turn, hybrid search can return the right result.
```
search_results = vi_client.hybrid_search(collection_name, 'R170NZKAXSA',
text_encoder.encode('R170NZKAXSA'),
fields=['name_vector_'], text_fields=['name'],
traditional_weight=0.015,
page_size=3)
vi_client.show_json(search_results, selected_fields=['_id', 'name', 'sku'],
image_fields=['image_url'], image_width=150)
```
From above, we realize that the value of hybrid search allows us to match items/products when exact values are known by the searcher.
If you are interested in exploring documentation around hybrid search, you can find that [here.](https://vector-ai.github.io/vectorai/vector_search.html?highlight=hybrid_search#vectorai.api.search.ViSearchClient.hybrid_search)
# Personalisation with vector search
An introduction to search personalisation with vectors
**Assumed Knowledge**: Vectors, Vector Search\
**Target Audience**: Data scientist, Vector enthusiasts, Business analysts, Executives\
**Reading Time:** 3 minutes
Vectors provide a simple way of personalising search and tailoring these searches for the individual. As is proposed in the original [Word2Vec](https://arxiv.org/pdf/1301.3781.pdf) paper (Mikolov, T., Chen, K., Corrado G., Dean J.), vectors have relationships. The famous example given is:
When we consider the following example, when we subtract the man vector from the king vector and add it to the woman vector, the closest vector is the queen vector (when we exclude the other vectors used to form the equation).
$$
King - Man + Woman = Queen
$$
This is very useful as this means that vectors in the vector space hold relationships This famous example can be extended into how we can use vectors for **personalisation.**
Let us extend this to a **retail use case study** to better understand how personalisation works with vectors.
**Simplified Retail Use Case**
Let us consider a blender. An individual may have been shopping for a lot of blenders and have clicked on a lot of Samsung labeled blenders.
If we decide to take the average of their vectors, we will associate this used with the Samsung Blender vector.
The individual may then decide to search for kettles. Based on our previous searches for Samsung blenders, we may then decide to personalise his search towards Samsung kettles. This is done using a similar equation as above:
From this, we subtract the Samsung blender from the blender category and add it to the kettle category to get the Samsung Kettle. The Samsung kettle is therefore a personalised search result based on the user's preference of Samsung blenders.
# Personalised search/recommendations with vector search
Advanced Search By Positive/Negative IDs
**Recommendations**
Personalization with vector search can be done using the `/collection/searchbypositiveandnegative_ids endpoint` . This endpoint can be found here: .
Recommendation by retrieving the vectors from the specified list of positive and negative ids documents. Then performing a search with an aggregated vector that is the sum (depends on vector\_operation) of positive id vectors minus the negative id vectors.
**Search**
If you are looking to perform personalised search, you will be wanting to use this endpoint:
{% embed url="" %}
Simply include the positive and negative IDs as before and then add a vector to search.
# Chunk search
A tutorial to chunk search in Vector AI
Chunk search provides us a way to search through parts of a document. For example - you may have a paragraph of text. You can then split up that paragraph of text into sentences and then create a `chunkvector` out of each. You can then search through the chunkvectors to obtain the most relevant part of that paragraph for your needs.
Chunkvectors can be useful for improving the explainability of search. If you are interested in testing chunk search out, there are a few endpoints (all with guides written to them and use cases).
* [chunk search](/vector-search/searching-with-vector-ai/chunk-search/how-to-chunk-search)
* [advanced chunk search](/vector-search/searching-with-vector-ai/chunk-search/how-to-do-multivector-chunk-search)
* [multi step chunk search](/vector-search/searching-with-vector-ai/chunk-search/how-to-do-multi-step-chunk-search)
# How To Chunk Search
A guide on how to use chunk search
The chunk search endpoint can be found at `/collection/chunk_search` . This chunk search endpoint is useful if you are looking to build more explainable searches by explaining which part of the text is most relevant for your use case.
The `chunk_search` endpoint is
# How To Do MultiVector Chunk Search
A guide to combining multiple vectors for chunk search
Sometimes we may want to add multiple vectors to our chunk search.
For this, we can simply use the `/collection/advanced`\_`chunk_search` endpoint. We simply switch out the vector argument and change it into a multivector\_query.
An example of constructing a chunkvector multi-vector query can be found below:
```
multivector_query = {
"chunkvector_1": {"vector": [0.2, 0.5, 0.1], "fields": ["test_chunkvector_"]},
"chunkvector_2": {"vector": [0.5, 0.2, 0.001], "fields": ["test_chunkvector_"]},
}
```
The only difference between constructing a multivector query for advanced chunk search and advanced search is that the fields will contain `chunkvector` instead of `vector`.
# How to do multi step chunk search
A tutorial on multi-step chunk search
Multi step chunk search can be useful for users who are:
* Trying to accelerate chunk search
* Want to capture meaning in one field and then explain in another field
* Want to match semantics in the field at a broader context-level and then hone in at the chunk level.
Multi-step chunk search can be useful in improving the explainability of general search and are now combined into 1 search endpoint for users.
The multistep chunk search endpoint can be found at `/collection/advanced_multistep_chunk_search`~~.~~
# 🧍How to diversify search results
Diversifying Search Results with Vector AI
Sometimes when we search, we are not only looking for the best search but also include search results/recommendations from a variety of results. This is helpful when we are looking into giving users/customers more diversity. The main endpoint that is being used for this is `post`\_`cluster_search.` Try out the endpoint easily on Playground!
{% embed url="" %}
# Clustering
Introduction To Clustering
**Assumed Knowledge**: Vectors, Vector Search\
**Target Audience**: Data scientist, Vector enthusiasts, Business analysts, Executives\
**Reading Time:** 3 minutes
**Clustering**: Interpret these vectors and your data to different buckets and interpret them more easily when combined with aggregation. Clustering provides a simple way to group vectors and metadata so that those with similar properties can be easily interpreted.
When we cluster, we can identify the key attributes and have a new way to observe our data.
If you are interested in an example of clustering, take a quick look at our playground:
{% embed url="" %}
In this dataset, we grouped together similar players with similar statistics. From here, we can tell which players are actually similar and new types of players that can emerge.
We take the player closest to the center of the cluster in order to represent the cluster by name only. From there, we take the average of the cluster and note the general statistics of the group. We can make comparisons to groups of players similar to Chad Prince and compare them to players like Simon Elliott.
# Clustering Vectors From Deep Learning models
How to cluster deep learning vectors
**Assumed Knowledge**: Vectors, Vector Search\
**Target Audience**: Data scientist, Vector enthusiasts, Business analysts, Executives\
**Reading Time:** 3 minutes
Clustering vectors from deep learning models can provide interesting insights into unstructured data. For example, you can take a look at and navigate the playground until you find "Image Clustering".
{% embed url="" %}
**Applications of Clustering**
Clustering lets you access and understand your information automatically without having to manually sort through every single photo to understand the different types of images you may have. Not only can these images be clustered but they can also be automatically tagged to suggest the type of category these photos may belong to and what they may be associated with.
# Aggregation
Aggregate your data based on clusters to better understand your data!
Aggregating your data with Vector AI can provide users with new ways to try out things with data.
We are interested in clustering data/vectors and then determining the average/max/total of that cluster. This is achieved via aggregation. Vector AI helps provides aggregation endpoints that:
* Aggregate live every time new data comes in
* Provide common statistical ways to aggregate clusters
* Using the centroid data point to represent the cluster
# Writing Your First Aggregation
A guide to writing your first aggregation
Aggregations refers to when you group a collection by a certain field and then take a metric when it is grouped.
An example of an aggregation query can be found as below:
```
{
"groupby" : [
{"name": ,
"field": , "agg": "category"},
{"name": ,
"field": ,
"agg": "numeric"}
],
"metrics" : [
{"name": ,
"field": ,
"agg": "avg"}
{
"name": ,
"field": ,
"agg": "max"}
]
}
```
The available groupby types are:
* category (groupby a field that is a category)
* numeric (groupby a field that is numeric)
Metrics are the fields you want to calculate in each of these every aggregation includes a frequency metric. These are the available metric types:
* average
* max
* min
* sum
* cardinality
To help memorise the aggregation queries, the following example can help:
```
{
"groupby" : [
{"name": "region", "field": "player_region", "agg": "category"},
{"name": "player_name", "field": "name", "agg": "category"}
],
"metrics" : [
{"name": "average_score", "field": "final_score", "agg": "avg"},
{"name": "max_score", "field": "final_score", "agg": "max"},
{'name':'total_score','field':"final_score", 'agg':'sum'},
{'name':'average_deaths','field':"final_deaths", 'agg':'avg'},
{'name':'highest_deaths','field':"final_deaths", 'agg':'max'},
]
}
```
In this example, we aggregate by 2 separate categories (region and player name). Based on each category, the average, max, sum of scores is calculated and the average and maximum number of deaths are calculated.
# Publishing Your First Aggregation
A guide on publishing your first aggregation
Now that you have worked out how to write an aggregation query, you may sometimes want to publish this query to get live updates of your results. This can be useful if you are interested in identifying "trending" categories and to automatically update "hottest" categories.
To publish an aggregation, you will want to use the [publish aggregation](https://vectorai-development-api.azurewebsites.net/collection/publish_aggregation) endpoint. Publishing an aggregation means that the data is aggregated into a new collection.
After you publish an aggregation, you may want to update the collection live. To do that, you wil want to use the `/collection/start_aggregation` endpoint. If you no longer want live updates and to update manually, you will want the `/collection/stop_aggregation` endpoint.
Finally if you can delete a published aggregation by using the `/collection/delete_aggregation` endpoint.
# Experimentation
# Vector Evaluation
An introduction to evaluating vectors
# Evaluate Vector Bias
A guide on evaluating vectors on whether they are biased.
**Assumed Knowledge**: Vectors\
**Target Audience**: Data Scientists, Vector Enthusiasts, Python Developers\
**Reading Time:** 3 minutes
To identify bias in the representation space, we want to know which direction vectors will be leaning towards. This can be achieved using **Normed 2D Cosine Similarity Plots**.
We will explore below how the model interprets certain terms and their bias. Let us consider the following male vs female comparison and then explore what these charts show.
### Male Vs Female
In the male vs female comparison:
Let us go over the main takeaways from this chart:
* The purple bar suggests that a certain is more biased towards "female" whereas the green bar suggests the word is more biased towards the "male".
* The words "princess", "skirt", "perfume" and "make-up" are all strongly tied to females.
* Comparatively, the words "computers", "football", "machine", "beer" and "prince" are all strongly tied to males.
* The magnitude of the cosine similarities are also interesting as it indicates that princess is more strongly tied to "female" in contrast to "skirt", "perfume", or "makeup". Conversely, "prince" is more strongly tied to "male" in contrast to "beer" or "machine".
In better understanding the hidden bias in our models, we may want to finetune these vectors and models.
**Analysing Between Groups**
The use cases of vectors can be extended even further. For example - when optimising for retail, we may want to decide where to place items and where to place categories such that customers can intuitively go to a section, find what they need and optimise conversion. For this, we will be interested in seeing where we should place each item and in which section. This can be optimised using bias indicator to determine where a particular item should go.
In the above graph, we explore a similar look into the representation space - looking over how different home tools compare to different categories. The above example compares technology and home gardening to different objects.
* Telephones, televisions, tablets, PCs, phone are more biased to technology than home gardening.
* Manure and garden hose are more biased to technology than home gardening.
* Cultivator is slightly more biased to home gardening compared to technology. This may be because while cultivators are useful for gardening (not necessarily just home gardening), they are more used for farms and are a product of technology.
### Using The Bias Indicator In VectorAI
```
from vectorai import ViClient
vi.bias_indicator(anchor_docs, docs, metadata_field='word')
```
This guide would not have been possible without the work of the following papers and articles by teams that have open-sourced their work for research purposes and for us to improve on.
```
@inproceedings{
author = {Piero Molino, Yang Wang, Jiwei Zhang},
booktitle = {ACL},
title = {Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae},
year = {2019},
}
```
# Jobs
Vector AI has a number of jobs to improve how documents and jobs work.
Vector AI provides ways to also try out and test features as they come up. These features are vector-based features that allow for a number of new applications. These include:
| Job Name / Endpoint | Description |
| ------------------- | ----------------------------------------------------- |
| encode image field | Encode a field which has images |
| | |
| tag | Tag a text field with possible tags from a dictionary |
# Tagging Jobs
A guide on how to run a tagging job
Tagging provides you with an automatic way to get concepts and associated metadata in relation to your data. For example, when you write images, you can get tags associated with your images. Similarly, when you write sentences/paragraphs/blogs/headings, you can get associated tags with what you are writing.
You can run a tagging job using the [/collection/job/tag\_job](https://vectorai-development-api.azurewebsites.net/collection/job/tag_job) endpoint.
Note: the [tag\_job](https://vectorai-development-api.azurewebsites.net/collection/job/tag_job) endpoint is different from the [/collection/job/tag\_vector\_job](https://vectorai-development-api.azurewebsites.net/collection/job/tag_vector_job) endpoint as this consists of
You simply require the following parameters with a POST request:
```
{
"username": "string",
"api_key": "string",
"collection_name": "string",
"tag_collection_name": "string",
"field": "",
"encoder_task": "text",
"tag_field": "tag",
"tag_vector_field": "",
"alias": "default",
"metric": "cosine",
"number_of_tags": 5,
"include_tag_vector": true,
"refresh": false,
"hub_username": "string",
"hub_api_key": "string",
"store_to_pipeline": true
}
```
# Chunking Jobs
# Encoding Jobs
# 🧍List all jobs (active and inactive)
List all active jobs available.
You are able to see all your jobs from the playground. Simply navigate to a chosen collection from the `collections` page and click "Jobs".
In order to list all active jobs available, you will want to use the [\`collection/job/list\_collection\_jobs](https://vectorai-development-api.azurewebsites.net/collection/job/list_collection_jobs)\` endpoint*.* This allows one to quickly see what jobs are currently running. If you prefer to see all jobs instead of simply running ones, simply change the argument for `show active jobs` = False.
# Encoding
Learn the different ways Vector AI can help you encode complex, unstructured data
Vector AI has deployed a few encoders that follow a very standard encoding scheme. This is to assist users with encoding as easily as possible.
These encoders are summarised in the table below.
| Encoder Name | Description |
| :----------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| text | Encode English text data into a vector. |
| text- multi | Encode multi-lingual text data into a vector. However, if you are looking to get good English vectors, we recommend the text encoder as it has better performance. |
| image | Encode an image. |
| image-text | Encode an image to make it searchable with text. However, if you are looking for good image similarity, we recommend the image encoder for better performance. |
| text-image | Encode text to make it searchable with images. However, If you are looking for text similarity, we recommend the text encoder for better performance. |
# Maintenance & Monitoring
A guide to handling Vector AI on a project basis
Vector AI makes various maintenance and monitoring features easy. For project/collection maintainers, you can:
* Give users read-only API keys
* Copy collections from other users easily
* Backup collections
* View various collection schemas
* Issue new API keys
# How to view your collections
A guide on how to view your collections in Vector AI Playground
The Vector AI playground provides an easy way to view all your collections as well as a summary of the information around your collections.
{% embed url="" %}
# How to share your collections
A guide to sharing your collections with Vector AI
Vector AI is focused on ensuring that data remains secured even when sharing.
You can securely share data with other accounts/users using the following endpoint:
{% tabs %}
{% tab title="Python" %}
```python
import requests
response = requests.post(
url="https://vectorai-development-api.azurewebsites.net/project/copy_collection_from_another_user",
json={
"collection_name": "string",
"username": "string",
"api_key": "string",
"source_collection_name": "string",
"source_username": "string",
"source_api_key": "string"
})
```
{% endtab %}
{% tab title="JavaScript" %}
```javascript
let request = await fetch("https://vectorai-development-api.azurewebsites.net/project/copy_collection_from_another_user",
{
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"collection_schema": { },
"rename_fields": { },
"remove_fields": [ ]
})
}
)
let json = await request.json()
```
Ensure that you are using write API keys for both usernames and collections.
{% endtab %}
{% endtabs %}
# How to back up your collections
A guide on backing up your collections
You can back up your collections easily using the "copy\_collection" endpoint. As you back up your collection, you may also choose to rename the fields, remove the fields or change the collection schema. All these endpoints allows users to rename the required fields.
{% tabs %}
{% tab title="Python" %}
```python
import requests
response = requests.post("https://vectorai-development-api.azurewebsites.net/project/copy_collection",
json={
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"collection_schema": { },
"rename_fields": { },
"remove_fields": [ ]
})
response.json()
```
{% endtab %}
{% tab title="JavaScript" %}
```javascript
let request = await fetch("https://vectorai-development-api.azurewebsites.net/project/copy_collection", {
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"collection_schema": { },
"rename_fields": { },
"remove_fields": [ ]
})
})
let json = await request.json()
```
{% endtab %}
{% endtabs %}
# How to change name of a collection field
How to change the name of a collection field
Changing the name of a collection field requires us to make use of the `/project/copy_collection` endpoint and in particular the `rename_fields` argument.
This can be seen as:
{% tabs %}
{% tab title="JavaScript" %}
```javascript
let request = await fetch("https://vectorai-development-api.azurewebsites.net/project/copy_collection", {
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"rename_fields": { "old_field_name": "new_field_name" },
})
})
let json = await request.json()
```
{% endtab %}
{% tab title="Python" %}
```python
import requests
response = requests.post("https://vectorai-development-api.azurewebsites.net/project/copy_collection",
json={
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"collection_schema": { },
"rename_fields": { "old_field_name": "new_field_name" },
"remove_fields": [ ]
})
response.json()
```
{% endtab %}
{% endtabs %}
# How to change the schema of a collection
How to change the schema of a collection
We can change the schema of a collection using the `/project/copy_collection` endpoint and fill in the value for the `collection_schema` argument.
{% tabs %}
{% tab title="Python" %}
```python
import requests
response = requests.post("https://vectorai-development-api.azurewebsites.net/project/copy_collection",
json={
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"collection_schema": { "field_vecotr_": {"vector" : {"vector_length" : 256}} },
})
response.json()
```
{% endtab %}
{% tab title="JavaScript" %}
```javascript
let request = await fetch("https://vectorai-development-api.azurewebsites.net/project/copy_collection", {
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"collection_schema": { "field_vector_": {"vector" : {"vector_length" : 256}} },
})
})
let json = await request.json()
```
{% endtab %}
{% endtabs %}
# How to remove a field in a collection
How to remove a field in a collection
We can change the schema of a collection using the `/project/copy_collection` endpoint and fill in the value for the `remove_fields` argument.
{% tabs %}
{% tab title="Python" %}
```python
import requests
response = requests.post("https://vectorai-development-api.azurewebsites.net/project/copy_collection",
json={
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"remove_fields": ["field_to_remove"]
})
response.json()
```
{% endtab %}
{% tab title="JavaScript" %}
```javascript
let request = await fetch("https://vectorai-development-api.azurewebsites.net/project/copy_collection", {
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"username": "string",
"api_key": "string",
"collection_name": "string",
"original_collection_name": "string",
"remove_fields": ["field_to_remove"]
})
})
let json = await request.json()
```
{% endtab %}
{% endtabs %}
# How to request a read API key
A guide on how to request a read API key
If you are looking to provide a **read-only** API key, then Vector AI has an endpoint to allow users to simply get one.
You simply need the following endpoint: `/project/request_read_api_key`
{% tabs %}
{% tab title="Python" %}
```python
import requests
response = requests.post("https://vectorai-development-api.azurewebsites.net/project/request_read_api_key",
json={
"username": "string",
"api_key": "string",
"read_username": "string"
})
response.json()
```
{% endtab %}
{% tab title="JavaScript" %}
```javascript
let request = await fetch("https://vectorai-development-api.azurewebsites.net/project/request_read_api_key", {
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"username": "string",
"api_key": "string",
"read_username": "string"
})
})
let json = await request.json()
```
{% endtab %}
{% endtabs %}
# How to turn data into Vectors (code)
A guide to turning data into vectors with VectorHub.
**Assumed Knowledge**: Vectors\
**Target Audience**: Python developers, General developers\
**Reading Time:** 3 minutes
The process of turning structured/unstructured data (in the form of Excel Spreadsheets, videos, images, word documents, PDFs) into vectors can involve quite complicated pipelines.
To help transform data into vectors, we open-sourced a library called **VectorHub** (you can explore the hub at hub.vctr.ai). For this, you will need to use Python 3 (tested on Python 3.6/Python 3.7).
The library can be installed via pip:
```
$ pip install vectorhub
```
Once you install via pip, you can then use a model in Python. For example:
```
$ from vectorhub.encoders.text import ViText2Vec
```
You can easily instantiate the model using the below.
```
from vectorai import request_api_key
username = input("What is your username")
email = input("What is your email?")
api_key = request_api_key(username, email, description="Trying out VectorHub.")
vi = ViText2Vec(username, api_key)
```
Transforming your data into vectors is as simple as the following:
```
text = "My dog loves taking long walks on the beach!"
vector = vi.encode(text)
# Voila, you have your vector!
```
**What is happening under the hood in VectorHub?**
In this library, VectorHub abstracts away a few complexities to make the encoding smooth. There is, however, still a lot of room for customisation.
Quickly going over the diagram, as data is parsed through VectorHub, it is converted into a NumPy array. This array is fed through the model, pooled together and is then transformed into a native Python object.
For each of the different data types, we have sections on how each of them are converted into vectors so users can understand what is occurring in each of the processes.
These pages can be found below if you are interested:
{% content-ref url="/pages/-MTOMPQFpl-cIriHlK2R" %}
[How to turn images Into Vectors](/vector-ai-documentation/how-to-turn-data-into-vectors-code/how-to-turn-images-into-vectors)
{% endcontent-ref %}
{% content-ref url="/pages/-MTOMHrEj0E5x\_waAD6W" %}
[How to turn audio into Vectors](/vector-ai-documentation/how-to-turn-data-into-vectors-code/how-to-turn-audio-into-vectors)
{% endcontent-ref %}
{% content-ref url="/pages/-MTOBP4sbQqhH527aL9T" %}
[How to turn text into Vectors](/vector-ai-documentation/how-to-turn-data-into-vectors-code/how-to-turn-text-into-vectors)
{% endcontent-ref %}
# How to turn text into Vectors
An introduction to turning data into vectors.
**Assumed Knowledge**: Vectors\
**Target Audience**: Python developers, general developers\
**Reading Time:** 3 minutes
To help transform data into vectors, we open-sourced a library called **VectorHub** (you can explore the hub at hub.vctr.ai). For this, you will need to use Python, and you can run all of the below on Colab.
The library can be installed via pip:
```
$ pip install vectorhub[encoders-text-tfhub]
```
Once you install via pip, you can then use a model in Python. For example:
```
from vectorhub.encoders.text.tfhub import USE2Vec
enc = USE2Vec()
text= "How do you encode data?"
vector = enc.encode(image_url)
```
From this - you will have obtained a vector which can now be indexed and stored away for search. If you are interested in reading what is occurring under the hood or to write your own library for this - take a look below.
**What is occurring under the hood?**
We vectorise a sentence by firstly tokenizing text into separate subwords (these are called tokens), each token is mapped to a separate vector which is then fed through the model. Note: models are not necessarily trained for the best vectors and representation space and specific models will need to be identified for different use cases. If there is a use case you would like, feel free to message us in our [Discord](https://discord.gg/CbwUxyD).
# How to turn images Into Vectors
A guide to turning images into Vectors.
**Assumed Knowledge**: Vectors\
**Target Audience**: Python developers, general developers\
**Reading Time:** 3 minutes
To help transform data into vectors, we open-sourced a library called **VectorHub** (you can explore the hub at hub.vctr.ai). For this, you will need to use Python, and you can run all of the below on Colab.
The library can be installed via pip:
```
$ pip install vectorhub[encoders-image-tfhub]
```
Once you install via pip, you can then use a model in Python. For example:
```
from vectorhub.encoders.image.tfhub import BitMedium2Vec
enc = BitMedium2Vec()
image_url = "https://upload.wikimedia.org/wikipedia/commons/8/85/Elon_Musk_Royal_Society_%28crop1%29.jpg"
vector = enc.encode(image_url)
```
From this - you will have obtained a vector which can now be indexed and stored away for search. If you are interested in reading what is occurring under the hood or to write your own library for this - take a look below.
**What is occurring under the hood?**
We vectorise an image by firstly reading in an image, which is turned into an array, resized for the model and fed through the model to extract the vector. Note: models are not necessarily trained for the best vectors and representation space and specific models will need to be identified for different use cases. If there is a use case you would like, feel free to message us in our [Discord](https://discord.gg/CbwUxyD).
# How to turn audio into Vectors
**Assumed Knowledge**: Vectors\
**Target Audience**: Python developers, general developers\
**Reading Time:** 3 minutes
To help transform data into vectors, we open-sourced a library called **VectorHub** (you can explore the hub at hub.vctr.ai). For this, you will need to use Python, and you can run all of the below on Colab.
```
pip install vectorhub[encoders-audio-tfhub]
```
The following model reads in the audio file and then encodes it (turning it into a vector).
```
from vectorhub.encoders.audio.tfhub import Vggish2Vec
model = Vggish2Vec()
sample = model.read('https://vecsearch-bucket.s3.us-east-2.amazonaws.com/voices/common_voice_en_2.wav')
model.encode(sample)
```
From this - you will have obtained a vector which can now be indexed and stored away for search.
# Image Search For Developers
Introduction To Image Search For Developers
### Setup
{% tabs %}
{% tab title="Python SDK" %}
```python
from vectorai import ViClient
vi = ViClient()
```
{% endtab %}
{% tab title="Python API" %}
```python
import requests
requests.post(
```
{% endtab %}
{% endtabs %}
You can search images using the endpoint when encoding as below.
{% tabs %}
{% tab title="Python SDK" %}
```
search_results = vi_client.search(
collection_name,
text_encoder.encode('samsung phone'),
'name_vector_',
page_size=5,
page=1
)
```
{% endtab %}
{% tab title="Python API" %}
```
```
{% endtab %}
{% endtabs %}
### Image Search
You can search the images using the following
{% tabs %}
{% tab title="Python SDK" %}
```
vi_client.show_json(
search_results,
selected_fields=['_id', 'name', 'sku'],
image_fields=['image_url'],
image_width=100
)
```
{% endtab %}
{% endtabs %}
# How To Combine Different Vectors For Search
Searching across multiple vectors is easy with Vector AI!
**Required Knowledge**: Vectors, Encoding, Vector Search\
**Audience**: Data scientists, Vector enthusiasts, Statisticians, Machine learning engineers\
**Reading time**: 3 minutes
When we are searching, sometimes we may want to combine multiple vectors. Constructing the multivector query can be quite difficult at first so here, so here we show the idea behind these different types of multivector queries.
{% tabs %}
{% tab title="Python SDK" %}
```python
from vectorai import ViClient
vi = ViClient()
multivector_query = {
"semantic_search": {"vector": vector_1, "field": {"bert_vector_": 0.3}},
"bag_of_word_search": {"vector": vector_2, "field": {"bow_vector_": 0.7}}
}
vi.advanced_search(
collection_name=collection_name,
multivector_query=multivector_query)
```
{% endtab %}
{% endtabs %}
Breaking down the multivector query
The `semantic_search` and `bag_of_word_search` are aliases of the multivector query. These can be used to improve readability of the different search query constructions.
In the advanced search query, there are 2 required fields. `vector` and `field`. The `vector` field contains the vector and the `field` field contains an object where the key is the vector field and the value has the weight.
The `advanced_search_query` is designed to provide a flexible way to search collections. This can be helpful to provide a number of ways combine vector spaces if you want to combine the multilingual specialties of one vector space with the slang specialties of another vector space.
# How To Combine Different Vectors With Exact Matching Text
Searching across multiple vectors with exact text match
**Required Knowledge**: Vectors, Encoding, Vector Search, Advanced Search\
**Audience**: Data scientists, Vector enthusiasts, Statisticians, Machine learning engineers\
**Reading time**: 3 minutes
Ensure that you have read [advanced search queries](/vector-ai-documentation/400-how-to-combine-different-vectors-to-search) before continuing below.
Once you have constructed the multivector query, adding hybrid search (searching with exact text) is easy. Simply alter the endpoint to `advanced_hybrid_search` and then add 2 additional fields: the `text` and the `text_fields`.
```python
vi.advanced_hybrid_search(
collection_name=collection_name,
multivector_query=multivector_query,
text='sample_text',
text_fields=["sample_text_field"])
```
If you want to increase the weight of exact text matching, you can alter `traditional_weight` parameter.
# Semantic NLP search with FAISS and VectorHub
**Assumed Knowledge**: Vectors\
**Target Audience**: Data scientists, Python developers\
**Reading Time:** 3 minutes
The following guide uses VectorHub and FAISS (by Facebook) to show an example of how to use vectors for search.
**Step 0) Getting the right Python and requirements**
Here, we use Python3.6/Python3.7. We have tested the code on Colab to ensure that this works even if you do not have your own Python installed. If you are interested in running the code in Colab, click [here](https://colab.research.google.com/drive/1rn-s1DlXzIZ9EEN8W7H6KIpvufSxcIoV?usp=sharing).
#### Step 1) Encoding Data With Vectors
First, we install VectorHub to encode models easily. We install the `encoders-text-tfhub` extra requirement because we are interested in using VectorHub's Bert model. You can find more about the Bert model [here](https://arxiv.org/abs/1810.04805). Bert was a model released by Google that provides bi-directional encoding with attention layers that led to a significant improvement in NLP performance.
```
%%capture
!pip install vectorhub[encoders-text-tfhub]
```
Then, we want to instantiate our model and start encoding. VectorHub abstracts away the dependency requirements into simple installation steps like above and also uses the best model and default pooler strategies based on our own tests. You can read more about Bert2Vec on the VectorHub model card [here](https://hub.getvectorai.com/model/text%2Fbert).
```
from vectorhub.encoders.text.tfhub import Bert2Vec
bert_enc = Bert2Vec()
# Words
words = [
'How can I design my own post-graduate education?',
'How could water be produced on Mars?',
'How can I fall in love?',
'How can India improve in corruption?'
]
vectors = []
# This can be optimised using list comprehension but
#we make it easier to read just for demo purposes
for word in words:
vector = bert_enc.encode(word)
vectors.append(vector)
```
**Step 2) Building An Index**
We then add our vectors and their associated words to the FAISS index. The FAISS index can be instantiated in a number of different ways. In this case, we instantiate it with the L2 index and then add the models. As they require numpy arrays for compatibility reasons, we convert them to compatible numpy arrays before inserting them into the index.
```
import numpy as np
import faiss
vector_length = len(vector)
index = faiss.IndexFlatL2(vector_length) # build the index using L2 as the distance
index.add(np.array(vectors).astype('float32')) # add vectors to the index
```
**Step 3) Searching Our Index**
Once you build the index, you encode the query vector. From the query vectors, we locate the closest vectors to the query vector.
```
num_of_results = 3 # Number of results
# Search using
search_term = 'Building a better government'
query_vector = bert_enc.encode(search_term)
D, I = index.search(np.array([query_vector]).astype('float32'), num_of_results)
# Return the results in order
for i in range(k):
print(words[I[0][i]])
```
Voila! You have built a very basic semantic search with FAISS. From here, you may add more to the index, build improved search or use your own datasets. FAISS search, however, is limited in its ability to provide support for more advanced search options (searching with filters, multi-vector search, personalised search). For these additional requirements (as well as online storage), we recommend reading [Vector Search With Vector AI](/what-are-vectors/what-is-vector-search/vector-search-with-vector-ai) which is our cloud-based vector search solution.
# Credits
Page of credits
Various icons and images made by [Freepik](https://www.freepik.com) from [www.flaticon.com](https://www.flaticon.com/).\
Icons made by [Becris](https://www.flaticon.com/authors/becris) from [www.flaticon.com](https://www.flaticon.com/)
# Philosophy
Vector AI aims to make vectors accessible to everyone.
**Target Audience:** General Audience
Vector AI's mission is to make vectors an accessible technology for businesses. To do this, we build technology and write material that makes accessing and understanding vectors easier for others.
The Vector AI team is focused on making vectors accessible. We often build our own products on top of the Vector AI platform in order to assess the quality of the engine, how we can improve the developer experience and figure out ways to demonstrate vector search to larger audiences.
# Glossary
Glossary of language used throughout this book or guide
**Document** refers to a vector and its associated metadata. An example of a document:
```
document_example = {
"car": {
"wheels":
{
"number": 4
}
},
"car_image_vector_": [0.52, 0.36, 0.12]
}
```
* **Field**, as shown in the example above is the value used to access a specific document value. For example, in the example above, 'car' is a field and 'number' is a field, but 4 is not a field.
* **Value** refers to what is returned by the field. For example, "number" field refers to 4.