Using Geosite with Surge

Surge does not support Geosite rule sets directly: it uses wildcard-based domain matching rules. To use the comprehensive Geosite lists, they first need to be converted into a format Surge understands.

Many GitHub repositories use GitHub Actions to convert Geosite rule sets into Surge rules on a schedule. That means the Surge rules do not update as soon as Geosite changes, and always lag behind upstream. Several repositories I checked also simply discarded Geosite regex rules, losing a large number of matches in the process.

So I decided to build a Cloudflare Worker service that converts rules on demand and optimistically caches the upstream lists. I also wanted to handle as many regex rules as possible, with the aim of avoiding lost rules.

And so xxxbrian/Surge-Geosite was born, and I deployed a public service at surge.bojin.co.

Usage

Append the Geosite list name to the path. To filter by an attribute, append a filter in the form @属性.

https://surge.bojin.co/geosite/<列表名>[@<属性>]

apple covers all of Apple's domains, while apple@cn keeps only the portion marked with @cn upstream. In Surge, use the URL directly as a RULE-SET source.

[Rule]
RULE-SET,https://surge.bojin.co/geosite/apple@cn,DIRECT
RULE-SET,https://surge.bojin.co/geosite/category-ads-all,REJECT
RULE-SET,https://surge.bojin.co/geosite/openai,Proxy,update-interval=3600

You can get a JSON list of all available list names at surge.bojin.co/geosite.

Implementation

Surge's domain rules use the rule types DOMAIN, DOMAIN-SUFFIX, DOMAIN-KEYWORD, and DOMAIN-WILDCARD. You can use RULE-SET to reference a remote list.

Geosite rule sets, on the other hand, use domain:, full:, keyword:, regexp:, and include:, and a rule can also have something like @cn at the end to filter for a specific region.

The first few types of rules can be converted directly in a one-to-one mapping, but regexp: and include: are more troublesome.

One-to-One Rules

The implementation goes through each line, checks the prefix, and replaces it with the corresponding Surge rule output. The @cn attribute at the end of a line, along with any comments, is preserved in a comment such as # 注释 after the rule. This lets you see its upstream attributes when reading the rule in Surge. For example, converting apple looks roughly like this.

# include:apple-dev
# include:swift
DOMAIN-SUFFIX,swiftui.cn # @cn
DOMAIN-SUFFIX,swiftui.com.cn # @cn

include

include handling is recursive. When the Worker encounters include:xxx, it goes to raw.githubusercontent.com to fetch the corresponding list, convert it with the same logic, and paste it back in the original position, leaving a line # include:xxx above it as a marker. The final output is a complete list; Surge does not need to know about the upstream include relationships.

Regex

Surge's domain rules have no regex type; the closest is DOMAIN-WILDCARD, but DOMAIN-WILDCARD matches using wildcards and only supports * and ?, so the challenge is to approximate complex regexes using *.

First I tallied the upstream regexes. At the time there were 1,283 lists in total, and regexp: had 208 lines, of which category-porn accounted for 134 lines, qcloud 29 lines, apple 6 lines, and the rest were scattered. Not a single one was a pure literal, so none could be directly converted to DOMAIN or DOMAIN-SUFFIX. Most look like this; these three come from apple.

regexp:^a[1-5]\.mzstatic\.com$ @cn
regexp:^cl[1-5]\.apple\.com$ @cn
regexp:^s[1-5]?\.mzstatic\.com$ @cn

They describe CDN hostnames with different numbers, such as a1 to a5 and so on. These need to be represented with wildcards.

At first I directly replaced regexp: with DOMAIN-KEYWORD, putting the regex in unchanged, and only later realized that DOMAIN-KEYWORD only performs substring matching. Characters such as ^, [, and ( never appear in hostnames at all, so the converted rules could never match.

I also considered string replacement, but groups, alternations, character classes, quantifiers, and escapes can be nested inside one another, and handling it character by character makes it easy to make mistakes with parentheses. So in the end I used regjsparser to parse each regex into a syntax tree, then handled each kind of node separately.

import { parse } from "regjsparser";

const convertNodeToWildcard = (node): string => {
  switch (node.type) {
    case "alternative": return node.body.map(convertNodeToWildcard).join("");
    case "anchor":      return "";   // ^ 和 $
    case "disjunction": return "*";  // a|b
    case "quantifier":  return "*";  // x* x+ x? x{m,n}
    case "reference":   return "*";  // 反向引用
    case "group":       return convertNodeToWildcard(node.body[0]);
    case "dot":
    case "characterClass":
    case "characterClassEscape":
    case "unicodePropertyEscape": return "?"; // 恰好一个字符
    case "value":       // 字面字符
      return typeof node.codePoint === "number" ? String.fromCodePoint(node.codePoint) : "?";
  }
};

Here I merged several branches that return the same value. The rules can be summarized as: literal characters are preserved as-is; ^ and $ are removed directly, because wildcards match the whole string anyway; characters such as ., [a-z], and \d each match exactly one character and become ?; *, +, ?, and {m,n} are quantifiers and become *; the alternation a|b cannot be represented, so it also becomes *; the contents inside parentheses are processed recursively.

With these rules, the three above are converted like this.

a?.mzstatic.com
cl?.apple.com
s*.mzstatic.com

If regex parsing fails, the conversion function returns an empty string, and the output is an incomplete line of DOMAIN-WILDCARD,. Surge skips invalid lines in a rule set and gives a warning, without affecting the whole list, so I did not handle it separately.

Comparison with Actions

Compared with scheduled conversion by an Action, the update delay has been shortened from one day to about half an hour. Regexes are no longer simply discarded; instead, they are converted to DOMAIN-WILDCARD. Combinations of lists and attributes do not need to be generated in advance; apple@cn requests are converted only when they arrive, so you can have whichever one you want. Also, the output is not placed on GitHub but served directly by Cloudflare's edge nodes, so you don't need to worry about GitHub being slow or unreachable.

The service does not have much code; if you are interested, take a look at the repository. It also contains an Action that runs once a day, solely to update the upstream submodule and regenerate the list index in the README.