Quantcast
Channel: Changing Bits
Browsing latest articles
Browse All 52 View Live

A new Lucene suggester based on infix matches

Suggest, sometimes called auto-suggest, type-ahead search or auto-complete, is now an essential search feature ever since Google added it almost 5 years ago. Lucene has a number of implementations; I...

View Article



2X faster PhraseQuery with Lucene using C++ via JNI

I recently described the new lucene-c-boost github project, which provides amazing speedups (up to 7.8X faster) for common Lucene query types using specialized C++ implementations via JNI. The code...

View Article

A new version of the Compact Language Detector

It's been almost two years since I originally factored outthe fast and accurateCompact Language Detector from the Chromium project, and the effort was clearly worthwhile: the project is popular and...

View Article

SuggestStopFilter carefully removes stop words for suggesters

Lucene now has a nice set of suggesters that use an analyzer to tokenize the suggestions: AnalyzingSuggester, FuzzySuggester and AnalyzingInfixSuggester. Using an analyzer is powerful because it lets...

View Article

Three exciting Lucene features in one day

Three exciting Lucene features in one day Yesterday was a productive day: suddenly, there are three exciting new features coming to Lucene. Expressions module The first feature, committed yesterday, is...

View Article


Lucene now has an in-memory terms dictionary, thanks to Google Summer of Code

Last year, Han Jiang's Google Summer of Code projectwas a big success: he created a new (now, default) postings format for substantially faster searches, along with smaller indices. This summer, Han...

View Article

Playing a sound (AIFF) file from Python using PySDL2

Sometimes you need to play sounds or music (digitized samples) from Python, which really ought to be a simple task. Yet it took me a little while to work out, and the resulting source code is quite...

View Article

Image may be NSFW.
Clik here to view.

Pulling H264 video from an IP camera using Python

IP cameras have come a long ways, and recently I upgraded some old cameras to these new Lorex cameras (model LNB2151/LNB2153) and I'm very impressed. These cameras record 1080p wide-angle video at 30...

View Article


Image may be NSFW.
Clik here to view.

Fast range faceting using segment trees and the Java ASM library

In Lucene's facet module we recently added support for dynamic range faceting, to show how many hits match each of a dynamic set of ranges. For example, the Updated drill-down in the Lucene/Solr issue...

View Article


Geospatial (distance) faceting using Lucene's dynamic range facets

There have been several recent, quiet improvements to Lucene that, taken together, have made it surprisingly simple to add geospatial distance faceting to any Lucene search application, for example:...

View Article

Image may be NSFW.
Clik here to view.

Finding long tail suggestions using Lucene's new FreeTextSuggester

Lucene's suggest module offers a number of fun auto-suggest implementations to give a user live search suggestions as they type each character into a search box. For example, WFSTCompletionLookup...

View Article

Using Lucene's search server to search Jira issues

You may remember my first blog post describing how the Lucene developers eat our own dog food by using a Lucene search application to find our Jira issues. That application has become a powerful...

View Article

Image may be NSFW.
Clik here to view.

Testing Lucene's index durability after crash or power loss

One of Lucene's useful transactional features is index durability which ensures that, once you successfully call IndexWriter.commit, even if the OS or JVM crashes or power is lost, or you kill -KILL...

View Article


Choosing a fast unique identifier (UUID) for Lucene

Most search applications using Apache Lucene assign a unique id, or primary key, to each indexed document. While Lucene itself does not require this (it could care less!), the application usually...

View Article

Image may be NSFW.
Clik here to view.

A new proximity query for Lucene, using automatons

The simplest Apache Lucene query, TermQuery, matches any document that contains the specified term, regardless of where the term occurs inside each document. Using BooleanQuery you can combine multiple...

View Article


Image may be NSFW.
Clik here to view.

Scoring tennis using finite-state automata

For some reason having to do with the medieval French, the scoring system for tennis is very strange. In actuality, the game is easy to explain: to win, you must score at least 4 points and win by at...

View Article

Apache Luceneā„¢ 5.0.0 is coming!

At long last, after a strong series of 4.x feature releases, most recently 4.10.2, we are finally working towards another major Apache Lucene release! There are no promises for the exact timing (it's...

View Article


Where are my new blog posts?

Some of you have noticed that I'm not writing much in this blog lately. But fear not: exciting changes are still happening in Lucene, and I am still writing about them! It's just that most of what I...

View Article

Jirasearch 2.0 dog food: using Lucene to find our Jira issues

A few years ago I first built and released Jirasearch as a fun dog-food test case for the thin-wrapper Lucene server, to expose a powerful search UI over our Jira issues. This is a great showcase of a...

View Article

Apache Lucene 7.0 Is Coming Soon!

The Apache Lucene project will likely release its next major release, 7.0, in a few months! Remember that Lucene developers generally try hard to backport new features for the next non-major (feature)...

View Article

Image may be NSFW.
Clik here to view.

Lucene gets concurrent deletes and updates!

Long ago, Lucene could only use a single thread to write new segments to disk. The actual indexing of documents, which is the costly process of inverting incoming documents into in-memory segment data...

View Article


Image may be NSFW.
Clik here to view.

Lucene's near-real-time segment index replication

[TL;DR: Apache Lucene 6.0 quietly introduced a powerful new feature called near-real-time (NRT) segment replication, for efficiently and reliably replicating indices from one server to another, and...

View Article


Image may be NSFW.
Clik here to view.

Concurrent query execution in Apache Lucene

Apache Lucene is a wonderfully concurrent pure Java search engine, easily able to saturate the available CPU or IO resources on your server, if you ask it to. The concurrency model for a "typical"...

View Article

Image may be NSFW.
Clik here to view.

Apache Lucene performance on 128-core AMD Ryzen Threadripper 3990X

Almost a decade ago, I started running Lucene's nightly benchmarks, and have been trying with mixed success to keep them running every night, through the numerous amazing changes relentlessly developed...

View Article

Image may be NSFW.
Clik here to view.

Open-source collaboration, or how we finally added merge-on-refresh to...

The open-source software movement is clearly a powerful phenomenon.A diverse (in time, geography, interests, gender (hmm not really, not yet, hrmph), race, skills, use-cases, age, corporate employer,...

View Article

Browsing latest articles
Browse All 52 View Live




Latest Images