Hosting a web site in radosgw Posted on Tue 26 January 2016 in hints-and-kinks • 6 min read

If you’re familiar with web site hosting on Amazon S3, which is a simple and cheap way to host a static web site, you might be wondering whether or not you can do the same in Ceph radosgw.

The short answer is you can’t. Bucket Website is listed as Not Supported in the radosgw S3 API support matrix, and radosgw doesn’t have index document support either.

But the longer answer is that you can, provided you use radosgw in combination with a front-end load-balancer — which, as it happens, can add a few more bells and whistles, as well. You could probably do the same thing with nginx, Varnish, or Apache in a mod_proxy_balancer balancer setup, but in this example configuration, we’ll use HAProxy.

Getting started: the radosgw basics

Let’s take look at a simple radosgw configuration with virtual host support, such that you can access your buckets as either http://ceph.example.com/bucketname or http://bucketname.ceph.example.com:

[client.rgw.radosgw01] rgw_frontends = civetweb port=7480 rgw_dns_name = ceph.example.com rgw_resolve_cname = True Suppose we use s3cmd to upload an HTML file to this bucket, setting a public ACL:

s3cmd mb s3://testwebsite s3cmd put –acl-public index.html s3://testwebsite/ Then if you exposed your radosgw to the web, any client (without authentication) would be able to retrieve http://testwebsite.ceph.example.com:7480/index.html with a web browser, or any other HTTP client application (such as curl or wget):

curl -I http://testwebsite.ceph.example.com:7480/index.html Which would then return something like:

HTTP/1.1 200 OK Content-Length: 18050 Accept-Ranges: bytes Last-Modified: Mon, 25 Jan 2016 21:28:47 GMT ETag: “b03130a4a1fc24df0f9f336f2b6d1d90” x-amz-request-id: tx000000000000000005a88-0056a7b7eb-312df-default Content-type: text/html Date: Tue, 26 Jan 2016 18:16:11 GMT Introducing HAProxy

Now let’s start out with putting HAproxy in between. Nothing special there: radosgw listens on the conventional 7480 port, and we simply hand HAproxy traffic through there, and bind HAProxy itself to port 80.

global log /dev/log local0 pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon

turn on stats unix socket #

stats socket /var/lib/haproxy/stats level admin

Default SSL material locations #

ca-base /etc/ssl/certs crt-base /etc/haproxy/ssl

Default ciphers to use on SSL-enabled listening sockets. #

For more information, see ciphers(1SSL). #

ssl-default-bind-ciphers HIGH tune.ssl.default-dh-param 2048

defaults log global mode http option httplog option dontlognull retries 3 timeout queue 1000 timeout connect 1000 timeout client 30000 timeout server 30000 option forwardfor

frontend ceph_front bind 0.0.0.0:80 default_backend ceph_back

backend ceph_back balance source server radosgw01 127.0.0.1:7480 check Index documents

So, the first thing we’ll need to add is support for index documents. We’d like to make sure that when we retrieve https://testwebsite.ceph.example.com/, what’s actually fetched from the backend is /index.html. We can do that by adding an HAproxy ACL that matches for the trailing slash in the path, and an http-request set-path directive that appends the index document name:

frontend ceph_front bind 0.0.0.0:80 acl path_ends_in_slash path_end -i /

Append index document (index.html) to any path #

ending in “/”. #

http-request set-path %[path]index.html if path_ends_in_slash default_backend ceph_back Now, that’s fine in terms of getting the index document correctly:

curl -I http://testwebsite.ceph.example.com/index.html HTTP/1.1 200 OK Content-Length: 18050 Accept-Ranges: bytes Last-Modified: Mon, 25 Jan 2016 21:28:47 GMT ETag: “b03130a4a1fc24df0f9f336f2b6d1d90” x-amz-request-id: tx000000000000000005a94-0056a7b9e3-312df-default Content-type: text/html Date: Tue, 26 Jan 2016 18:24:35 GMT However, it of course breaks uploads and even bucket listings, or in other words, anything that uses the S3 API. Now you could test for some S3-specific headers in the request, but really, you should just check whether the request is authorized, and only apply the index document logic if it isn’t, like so:

frontend ceph_front bind 0.0.0.0:80 acl path_ends_in_slash path_end -i / acl auth_header hdr(Authorization) -m found

Append index document (index.html) to any path #

ending in “/”, unless the request has an auth header #

http-request set-path %[path]index.html if path_ends_in_slash !auth_header default_backend ceph_back Great. Now we can upload using full paths without mangling, and on any un-authenticated requests, we substitute /index.html for any trailing /. In case you’re wondering: yes, this works for any path, not just the root path.

Directory paths

However, you may also want something else, which is the ability to correctly handle a request like http://testwebsite.ceph.example.com/my/sub/directory, where of course you want the path /my/sub/directory translated into /my/sub/directory/index.html, which means we want to append a slash and an index document name to the request path.

So let’s do that:

frontend ceph_front bind 0.0.0.0:80 acl path_has_dot path_sub -i . acl path_ends_in_slash path_end -i / acl auth_header hdr(Authorization) -m found http-request set-path %[path]index.html if path_ends_in_slash !auth_header

Append trailing slash if necessary. #

http-request set-path %[path]/index.html if !path_has_dot !path_ends_in_slash !auth_header default_backend ceph_back Note that what we’re doing here is somewhat crude. We’re assuming that any actual file that we want to retrieve looks like name.ext, meaning it has a dot (period, full stop) character in it. The path_sub -i . expression in the path_has_dot ACL simply matches any path with . in it, and we’re assuming that if a path has a dot then it points to a file, if it doesn’t then it points to a directory.

You could be a little more clever here and use path_regex instead of path_sub for a full regular expression match. But regex lookups are slower than simple substring matches, so if the substring match works for you, go for it.

So now, we can do this:

s3cmd put –acl-public index.html s3://testwebsite/my/sub/directory/ And then:

Note omitted trailing slash #

curl -I http://testwebsite.ceph.example.com/my/sub/directory HTTP/1.1 200 OK Content-Length: 24235 Accept-Ranges: bytes Last-Modified: Mon, 25 Jan 2016 23:57:04 GMT ETag: “fecd005b33c0f6bfdee61b787cf54cb0” x-amz-request-id: tx00000000000000000bc83-0056a7bd25-312cd-default Content-type: text/html Date: Tue, 26 Jan 2016 18:38:29 GMT HTTPS support

So, what else might you want to do? One obvious thing that you can use HAproxy for is SSL termination. The radosgw embedded civetweb webserver can do that for you, but that feature is currently mildly broken in a rather curious way. So in order to allow HTTPS access to all your content via HAproxy instead, you would add:

frontend ceph_front_ssl bind 0.0.0.0:443 ssl crt ceph.pem no-sslv3 no-tls-tickets reqadd X-Forwarded-Proto:\ https acl path_has_dot path_sub -i . acl path_ends_in_slash path_end -i / acl auth_header hdr(Authorization) -m found http-request set-path %[path]index.html if path_ends_in_slash !auth_header http-request set-path %[path]/index.html if !path_has_dot !path_ends_in_slash !auth_header default_backend ceph_back But maybe you’d like to force, not merely allow, HTTPS access. redirect to the rescue:

frontend ceph_front bind 0.0.0.0:80 reqadd X-Forwarded-Proto:\ http redirect scheme https code 301 if !{ ssl_fc }

Note HTTP #

curl -IL http://testwebsite.ceph.example.com/my/sub/directory HTTP/1.1 301 Moved Permanently Content-length: 0 Location: https://testwebsite.ceph.example.com/my/sub/directory Connection: close

HTTP/1.1 200 OK Content-Length: 24235 Accept-Ranges: bytes Last-Modified: Mon, 25 Jan 2016 23:57:04 GMT ETag: “fecd005b33c0f6bfdee61b787cf54cb0” x-amz-request-id: tx00000000000000000bdeb-0056a7bf9b-312cd-default Content-type: text/html Date: Tue, 26 Jan 2016 18:48:59 GMT Compression

And finally, maybe you’d like to speed up access to the stuff on your site. Why not add gzip on-the-fly-compression? It’s supported by every browser worth its salt, and will make your users happier. You’ll want to restrict compression to specific MIME types though. In the configuration below, we enable compression for plain text, HTML, XML, CSS, JavaScript, and SVG images.

frontend ceph_front bind 0.0.0.0:80 reqadd X-Forwarded-Proto:\ http redirect scheme https code 301 if !{ ssl_fc }

frontend ceph_front_ssl bind 0.0.0.0:443 ssl crt ceph.pem no-sslv3 no-tls-tickets reqadd X-Forwarded-Proto:\ https acl path_has_dot path_sub -i . acl path_ends_in_slash path_end -i / acl auth_header hdr(Authorization) -m found http-request set-path %[path]index.html if path_ends_in_slash !auth_header http-request set-path %[path]/index.html if !path_has_dot !path_ends_in_slash !auth_header compression algo gzip compression type text/html text/xml text/plain text/css application/javascript image/svg+xml default_backend ceph_back Let’s see how that helps us. Do a request without gzip encoding support, and observe that its total download size matches the document’s Content-Length:

curl https://testwebsite.ceph.example.com/my/sub/directory > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 24235 100 24235 0 0 94565 0 –:–:– –:–:– –:–:– 94299 Now, add an Accept-Encoding header:

curl -H ‘Accept-Encoding: gzip’ https://testwebsite.ceph.example.com/my/sub/directory > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5237 0 5237 0 0 19243 0 –:–:– –:–:– –:–:– 19324 There. Actual download size goes from 24KB down to just 5KB.

Where to go from here

There’s a few additional features to be added here. You could enable CORS or HSTS, for example, and of course you could add more backends. But if you read this far, you surely get the idea.

And you’re welcome to examine the headers you can pull from this page you’re reading, wink wink. :)

This article originally appeared on the hastexo.com website (now defunct).

Ceph