Configure Apache and Elasticsearch

Overview of secure web server communication

This topic discusses an example of securing communication between your web server and Elasticsearch using a combination of Transport Layer Security (TLS) encryption and HTTP Basic authentication. You can optionally configure other types of authentication as well; we provide references for that information.

(An older term, Secure Sockets Layer (SSL), is frequently used interchangeably with TLS. In this topic, we refer to TLS.)

Recommendations

We recommend the following:

• Your web server uses TLS.

  TLS is beyond the scope of this topic; however, we strongly recommend you use a real certificate in production and not a self-signed certificate.

• Elasticsearch runs on the same host as a web server. Running Elasticsearch and the web server on different hosts is beyond the scope of this topic.

  The advantage of putting Elasticsearch and the web server on the same host is that it makes intercepting encrypted communication impossible. The Elasticsearch web server doesn’t have to be the same as the Magento web server; for example, Magento can run Apache and Elasticsearch can run nginx.

More information about TLS

See one of the following resources:

• Apache

  • Apache 2.4 strong encryption how-to
  • Apache 2.2 SSL/TLS page
  • How To Create a SSL Certificate on Apache for Ubuntu 14.04 (Digitalocean tutorial)
  • Setting up an SSL secured Webserver with CentOS (CentOS wiki)

• nginx

  • nginx SSL termination
  • How To Create an SSL Certificate on Nginx for Ubuntu 14.04 (Digitalocean tutorial)
  • Nginx SSL Certificate Installation (digicert)

Set up a proxy

This section discusses how to configure Apache as an unsecure proxy so that Magento can use Elasticsearch running on this server. This section does not discuss setting up HTTP Basic authentication; that is discussed in Secure communication with Apache.

See one of the following sections:

• Set up a proxy for Apache 2.4
• Set up a proxy for Apache 2.2

Set up a proxy for Apache 2.4

This section discusses how to configure an Elasticsearch proxy using a virtual host.

1. Enable mod_proxy as follows:

   a2enmod proxy_http
   

2. Use a text editor to open /etc/apache2/sites-available/000-default.conf

3. Add the following directive at the top of the file:

   Listen 8080
   

4. Add the following at the bottom of the file:

   <VirtualHost *:8080>
      ProxyPass "/" "http://localhost:9200/"
      ProxyPassReverse "/" "http://localhost:9200/"
   </VirtualHost>
   

5. Restart Apache:

   service apache2 restart
   

6. Verify the proxy works by entering the following command:

   curl -i http://localhost:<proxy port>/_cluster/health
   

   For example, if your proxy uses port 8080:

   curl -i http://localhost:8080/_cluster/health
   

   Messages similar to the following display to indicate success:

   HTTP/1.1 200 OK
   Date: Tue, 23 Feb 2016 20:38:03 GMT
   Content-Type: application/json; charset=UTF-8
   Content-Length: 389
   Connection: keep-alive
   
   {"cluster_name":"elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":5,"active_shards":5,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":5,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":50.0}
   

7. Continue with Configure Magento to use Elasticsearch.

Set up a proxy for Apache 2.2

This section discusses how to configure an Elasticsearch proxy using a virtual host.

1. As a user with root privileges, open /etc/httpd/conf/httpd.conf in a text editor.

2. Locate the Listen directive and add another listen port; for example:

   Listen 8080
   

3. Scroll to the bottom of the file and add the following lines:

   <VirtualHost *:8080>
       ProxyPass http://localhost:9200/
       ProxyPassReverse http://localhost:9200/
   </VirtualHost>
   

4. Restart Apache:

   • CentOS: service httpd restart
   • Ubuntu: service apache2 restart

5. Verify the proxy works by entering the following command:

   curl -i http://localhost:<proxy port>/_cluster/health
   

   For example, if your proxy uses port 8080:

   curl -i http://localhost:8080/_cluster/health
   

   Messages similar to the following display to indicate success:

   HTTP/1.1 200 OK
   Date: Tue, 23 Feb 2016 20:38:03 GMT
   Content-Type: application/json; charset=UTF-8
   Content-Length: 389
   Connection: keep-alive
   
   {"cluster_name":"elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":5,"active_shards":5,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":5,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":50.0}
   

Configure Magento to use Elasticsearch

This section discusses the minimum settings you must choose to test Elasticsearch with Magento 2. For additional details about configuring Elasticsearch, see the Magento Commerce User Guide.

To configure Magento to use Elasticsearch:

1. Log in to the Magento Admin as an administrator.
2. Click Stores > Settings > Configuration > Catalog > Catalog > Catalog Search.

3. From the Search Engine list, click Elasticsearch or Elasticsearch 5.0+ as the following figure shows. (The Elasticsearch 5.0+ option is not available for Magento 2.1.)

   

4. The following table discusses only the configuration options required to test the connection with Magento.

   Unless you changed Elasticsearch server settings, the defaults should work. Skip to the next step.

   Option	Description
   Elasticsearch Server Hostname	Enter the fully qualified host name or IP address of the machine running Elasticsearch. Magento Commerce (Cloud): Get this value from your integration system.
   Elasticsearch Server Port	Enter the Elasticsearch web server proxy port. In our example, the port is 8080 but if you're using a secure proxy, it's typically 443. Magento Commerce (Cloud): Get this value from your integration system.
   Elasticsearch Index Prefix	Enter the Elasticsearch index prefix. If you use a single Elasticsearch instance for more than one Magento installation (Staging and Production environments), you must specify a unique prefix for each installation. Otherwise, you can use the default prefix magento2.
   Enable Elasticsearch HTTP Auth	Click Yes only if you enabled authentication for your Elasticsearch server. If so, provide a user name and password in the provided fields.

5. Click Test Connection.

One of the following displays:

Result	Meaning
	Magento successfully connected to the Elasticsearch server. Continue with Configure Apache and Elasticsearch or Configure nginx and Elasticsearch.
	Try the following: Make sure the Elasticsearch server is running. If the Elasticsearch server is on a different host from Magento, log in to the Magento server and ping the Elasticsearch host. Resolve network connectivity issues and test the connection again. Examine the command window in which you started Elasticsearch for stack traces and exceptions. You must resolve those before you continue. In particular, make sure you started Elasticsearch as a user with root privileges. Make sure that UNIX firewall and SELinux are both disabled, or set up rules to enable Elasticsearch and Magento to communicate with each other. Verify the value of the Elasticsearch Server Hostname field. Make sure the server is available. You can try the server's IP address instead. Use the command netstat -an | grep listen-port command to verify that the port specified in the Elasticsearch Server Port field is not being used by another process. For example, to see if Elasticsearch is running on its default port, use the following command: netstat -an | grep 9200 If Elasticsearch is running on port 9200, it displays similar to the following: tcp 0 0 :::9200 :::* LISTEN

Reindexing catalog search and refreshing the full page cache

After you change Magento’s Elasticsearch configuration, you must reindex the catalog search index and refresh the full page using the Admin or command line.

To refresh the cache using the Admin:

1. In the Admin, click System > Cache Management.
2. Select the check box next to Page Cache.
3. From the Actions list in the upper right, click Refresh.
   The following figure shows an example.
   

To clean the cache using the command line, use the magento cache:clean command.

To reindex using the command line:

1. Log in to your Magento server as, or switch to, the Magento file system owner.

2. Enter any of the following commands:

   Enter the following command to reindex the catalog search index only:

   php <your Magento install dir>/bin magento indexer:reindex catalogsearch_fulltext
   

   Enter the following command to reindex all indexers:

   php <your Magento install dir>/bin magento indexer:reindex
   

3. Wait while the reindexing completes.

Unlike the cache, indexers are updated by a cron job. Make sure cron is enabled before you start using Elasticsearch.

Secure communication with Apache

This section discusses how to secure communication between Apache and Elasticsearch using HTTP Basic authentication with Apache. For more options, consult one of the following resources:

• Apache 2.2 authentication and authorization tutorial
• Apache 2.4 authentication and authorization tutorial

See one of the following sections:

• Step 1: Create a password file
• Step 2: Configure your secure virtual host
• Verify communication is secure

Step 1: Create a password

For security reasons, you can locate the password file anywhere except your web server docroot. In this example, we show how to store the password file in a new directory.

Install htpasswd if necessary

First, see if you have the Apache htpasswd utility is installed as follows:

1. Enter the following command to determine if htpasswd is already installed:

   which htpasswd
   

   If a path displays, it is installed; if the command returns no output, htpasswd is not installed.

2. If necessary, install htpasswd:

   • Ubuntu: apt-get -y install apache2-utils
   • CentOS: yum -y install httpd-tools

Create a password file

Enter the following commands as a user with root privileges:

mkdir -p /usr/local/apache/password
htpasswd -c /usr/local/apache/password/.<password file name> <username>

where

• <username> can be:

  • Setting up cron: the web server user or another user.

    In this example, we use the web server user but the choice of user is up to you.

  • Setting up Elasticsearch: the user is named magento_elasticsearch in this example

• <password file name> must be a hidden file (starts with .) and should reflect the name of the user. See the examples later in this section for details.

Follow the prompts on your screen to create a password for the user.

Examples

Example 1: cron You must set up authentication for only one user for cron; in this example, we use the web server user. To create a password file for the web server user, enter the following commands:

mkdir -p /usr/local/apache/password
htpasswd -c /usr/local/apache/password/.htpasswd apache

Example 2: Elasticsearch You must set up authentication for two users: one with access to nginx and one with access to Elasticsearch. To create password files for these users, enter the following commands:

mkdir -p /usr/local/apache/password
htpasswd -c /usr/local/apache/password/.htpasswd_elasticsearch magento_elasticsearch

Add additional users

To add another user to your password file, enter the following command as a user with root privileges:

htpasswd /usr/local/apache/password/.htpasswd <username>

Step 2: Secure communication with Apache

This section discusses how to set up HTTP Basic authentication. Use of TLS and HTTP Basic authentication together prevents anyone from intercepting communication with Elasticsearch or with your Magento server.

This section discusses how to specify who can access the Apache server.

1. Use a text editor to add the following contents to your secure virtual host.

   • Apache 2.2: Depending on how you set up SSL, the Apache 2.2 SSL configuration might be located in /etc/httpd/conf/httpd.conf or /etc/httpd/conf.d/ssl.conf.

   • Apache 2.4: Edit /etc/apache2/sites-available/default-ssl.conf

     <Proxy *>
       Order deny,allow
       Allow from all
     
      AuthType Basic
      AuthName "Elastic Server"
      AuthBasicProvider file
      AuthUserFile /usr/local/apache/password/.htpasswd_elasticsearch
      Require valid-user
     
     # This allows OPTIONS-requests without authorization
      <LimitExcept OPTIONS>
        Require valid-user
      </LimitExcept>
      </Proxy>
     

2. If you added the preceding to your secure virtual host, remove Listen 8080 and the <VirtualHost *:8080> directives you added earlier to your unsecure virtual host.
3. Save your changes, exit the text editor, and restart Apache:

• CentOS: service httpd restart
• Ubuntu: service apache2 restart

Verify communication is secure

This section discusses two ways to verify that HTTP Basic authentication is working:

• Using a curl command to verify you must enter a user name and password to get cluster status
• Configuring HTTP Basic authentication in the Magento Admin

Use a curl command to verify cluster status

Enter the following command:

curl -i http://<host name, ip, or localhost>:<proxy port>/_cluster/health

For example, if you enter the command on the Elasticsearch server and your proxy uses port 8080:

curl -i http://localhost:8080/_cluster/health

The following message displays to indicate authentication failed:

HTTP/1.1 401 Unauthorized
Date: Tue, 23 Feb 2016 20:35:29 GMT
Content-Type: text/html
Content-Length: 194
Connection: keep-alive
WWW-Authenticate: Basic realm="Restricted"

<html>
<head><title>401 Authorization Required</title></head>
<body bgcolor="white">
<center><h1>401 Authorization Required</h1></center>
</body>
</html>

Now try the following command:

curl -i -u <user name>:<password> http://<host name, ip, or localhost>:<proxy port>/_cluster/health

For example:

curl -i -u magento_elasticsearch:mypassword http://localhost:8080/_cluster/health

This time the command succeeds with a message similar to the following:

HTTP/1.1 200 OK
Date: Tue, 23 Feb 2016 20:38:03 GMT
Content-Type: application/json; charset=UTF-8
Content-Length: 389
Connection: keep-alive

{"cluster_name":"elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":5,"active_shards":5,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":5,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":50.0}

Configure HTTP Basic authentication in the Magento Admin

Perform the same tasks as discussed in Configure Magento to use Elasticsearch except click Yes from the Enable Elasticsearch HTTP Auth list and enter your user name and password in the provided fields.

The following figure shows an example:

Click Test Connection to make sure it works and then click Save Config.

You must flush the Magento cache and reindex before you continue.

Next

Configure Elasticsearch stopwords