Hello everyone, I am using Redis to store vector embeddings, and would like to do a vector similarity search using the JedisClient. I have 1536 dimension embeddings , and I am storing the same embeddings in a HNSW field, as well as in a FLAT vector field. When I attempt to search for matches to this embedding using jedis.ftSearch(), I seem to get irrelevant documents with a vector_score of -nan Is there anything glaringly obvious that I am missing here?
I am using 4.3.0 from GitHub - redis/jedis: Redis Java client designed for performance and ease of use.
Index creation:
Schema schema = new Schema().addHNSWVectorField("hnsw_field", vectorHnswAttributes).addTagField("raw_text").addFlatVectorField("flat_field",vectorFlatAttributes);
uJedis.ftCreate(INDEX_NAME, IndexOptions.defaultOptions(), schema);
Search:
int K = 1; // Get K nearest neighbors
Query q_hnsw = new Query("*=>[KNN $K @hnsw_field $BLOB AS my_vector_score]").addParam("K", K).addParam("BLOB",floatToByte(vec))
.setSortBy("my_vector_score", false)
.limit(0,K).returnFields("my_vector_score","raw_text").dialect(2);
List<Document> docs_hnsw = uJedis.ftSearch(INDEX_NAME, q_hnsw).getDocuments();