Samba with macOS Spotlight on FreeBSD with Elasticsearch

Spotlight support on network shares in macOS simplifies searching through large collections of shared files. With gradual deprecation of AFP, here's how to set this up for SMB using samba on FreeBSD, with Spotlight support for macOS clients. I've tested this on FreeBSD 13.0-RELEASE with macOS 10.14 and 10.15 clients.

I'll be sharing the folder /sambashare as "share", accessible by a user named sambauser.

First, install samba and elasticsearch, as well as openjdk15 (dependency for fscrawler, minimum version is 15). You'll have to pkg search samba and elasticsearch to identify the most recent versions.

pkg install samba413 elasticsearch7 openjdk15

You'll want to configure elasticsearch by editing /usr/local/etc/elasticsearch/elasticsearch.yml. Take a look at the sample (elasticsearch.yml.sample) in the same directory. Mine looks like this:

cluster.name: samba
node.name: node1
path.data: /var/db/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 127.0.0.1 # Set to node IP address instead if access needed from another machine
http.port: 9200
xpack.ml.enabled: false
discovery.type: single-node # not in the sample file

Enable the elasticseach service by adding elasticsearch_enable=YES to /etc/rc.conf, then start it by running service elasticsearch start.

Set up samba by editing /usr/local/etc/smb4.conf. For shares that you want to index, add the directive spotlight = yes. Set the address of the elasticsearch server in the global config section. Here's my config file::

# /usr/local/etc/smb4.conf
[global]
server role = standalone server
workgroup = WORKGROUP
fruit:advertise_fullsync = true
fruit:aapl = yes
security = user
passdb backend = tdbsam
acl allow execute always = yes
spotlight backend = elasticsearch
elasticsearch:address = 127.0.0.1
elasticsearch:port = 9200

[share]
path = /sambashare
valid users = sambauser
writable = yes
browsable = yes
read only = no
guest ok = no
public = no
create mask = 0666
directory mask = 0755
vfs objects = catia fruit streams_xattr zfsacl
fruit:resource = file
fruit:encoding = native
spotlight = yes

Add samba_server_enable="YES" to /etc/rc.conf, and start samba with service samba_server start. I also had to set my user's SMB password with smbpasswd -a sambauser before I could successfully authenticate.

This configuration will allow samba to connect to elasticsearch. To actually index files, however, you'll also need to install fscrawler. The download page will have snapshots of fscrawler for various versions of elasticsearch — be sure to choose the right one. In my case, I downloaded the fscrawler-es7-2.7-20210426.095126-177.zip snapshot, and unzipped it to the home folder of my user with access to the samba share.

fscrawler is configured using YAML files, each one within a folder representing the task, all within the config folder. Here's my config directory in /home/sambauser/:

$ tree /home/sambauser/fscrawler
.
├── share
└── _settings.yaml

Here's my _settings.yaml. Take a look at all the options in the documentation.

---
name: "share"
fs:
  url: "/sambashare"
  update_rate: "15m"
  json_support: false
  filename_as_id: false
  add_filesize: true
  remove_deleted: true
  add_as_inner_object: false
  store_source: false
  index_content: true
  attributes_support: false
  raw_metadata: false
  xml_support: false
  index_folders: true
  lang_detect: false
  continue_on_error: true
  ocr:
    language: "eng"
    enabled: false
    pdf_strategy: "ocr_and_text"
  follow_symlinks: false
elasticsearch:
  nodes:
  - url: "http://127.0.0.1:9200"
  bulk_size: 100
  flush_interval: "5s"
  byte_size: "10mb"
  ssl_verification: false

Rather than running the indexer continuously, I run the following script (in /home/sambauser/index-filesystems.sh) to index my files:

#!/bin/sh
cd /home/sambauser/fscrawler-es7-2.7-SNAPSHOT/
JAVA_HOME=/usr/local/openjdk15 bin/fscrawler --config_dir /var/db/fscrawler share --restart --loop 1

I run the indexer daily via this line in /etc/crontab:

0 23 * * * sambauser /home/sambauser/index-filesystems.sh

With all this set up, I can now use the Find bar in the Finder when browsing a share to search the files within it with Spotlight.