Ardor

Ardor security incident: what happened on November 6–8, 2026

On November 6 and 8, 2026, Ardor experienced a security incident. A flaw in the handling of user application files allowed code to execute on the server. Secrets and service permissions available to that code enabled access to other customers' data, internal systems, and individual infrastructure hosts.

We detected the incident on November 8 and switched the system into emergency mode, disabling user applications and their external connections. We subsequently rebuilt production on a new cluster, moved web-page builds out of the API process, narrowed service permissions, and replaced platform credentials. During recovery, we checked that previous credentials were rejected. The response and recovery section describes these changes and their verification dates. This report explains how the attack unfolded, which consequences are confirmed, and which questions remain open.

Updated November 17, 2026. Recovery checks cited below were performed on November 12–16.

At a glance

  • What happened: on November 6 and 8, attackers used ordinary accounts to submit application files that became executable build configuration. The build process’s credentials and permissions then provided access to other services, customer data, and individual infrastructure hosts.
  • What was affected: recorded responses confirm disclosure of other users’ records, selected files and configuration, and service credentials. The impact section distinguishes confirmed disclosure from potential access and explains the limits of the available records.
  • What we did: after detection on November 8, we disabled user applications and external connections. We rebuilt production on a new cluster, separated web-page builds from the API process, restricted service permissions, and rotated platform credentials.
  • What remains restricted: some application build and deployment features remain temporarily restricted during additional security checks.

What customers should do

If your external-service keys were stored or used in an affected Ardor environment, treat them as potentially exposed: revoke and replace them at the services that issued them. We notified all users about this requirement. Ardor’s platform credential rotation does not replace this step. If you are unsure whether your workspace or credentials were affected, contact us for help with your specific case.

The sections below retain the detailed attack sequence, including failed probes, changes in approach, and the evidence supporting each confirmed step. The attacker-code release will follow separately.

Contents

1. How an application build became a server entry point

Ardor’s web-page build service accepts files supplied by a user and runs build tools to turn them into a web page. We call these applications and their build outputs artifacts. An artifact can have several versions, each containing a different set of files. The service must treat those files as untrusted code.

The flaw crossed the boundary between those files and the build tool's own configuration. A user could supply a path outside the source directory and write a configuration file into the build root. Vite, the web application build tool, then loaded that configuration and executed its JavaScript. Uploading files could therefore lead to server-side code execution.

The code had access to the server process environment and its service account. The environment contained configuration and secrets, while the service account provided access to internal systems. The consequences extended beyond one user application: the attackers could reach databases, other applications, and external services.

A separate path through the container image builder and privileged containers enabled root access, the highest level of local privilege, on individual worker hosts. This was a distinct stage: executing code inside a container and controlling its underlying host have different consequences.

Command results were returned through build error messages. Attacker-controlled code could read data or modify records, then deliberately fail the build and include the output in the error text. A failed build therefore did not mean the attack attempt had failed.

The technical chain: boundaries crossed

This chain developed through repeated probes and revisions. For the Ada Lovelace episode, the preserved records contain 153 distinct artifacts and 155 versions over approximately 21 minutes; for John Tester, 86 distinct artifacts, each with one preserved version, over approximately 14 minutes. That is 239 artifacts and 241 versions in total. The preserved records show that almost every successive attempt used a new artifact with revised code or commands. There is one preserved version for 238 artifacts and three versions for one artifact in the first episode. These are intervals of preserved build activity, not the full duration of the attack. The early probes tested configuration placement in the build root and source directory; later probes tested command execution and access to the environment, secrets, databases, and neighboring services. Attempts changed in response to results: decryption errors were followed by checks of other secret sources, and denial of an overly broad Kubernetes role was followed by a more limited role. One version could contain several operations; 241 versions do not mean 241 individual commands.

  1. Writing outside the source directory. An authenticated request to the build API supplied the path ../vite.config.js in the field FileInput.path. The server joined it to the directory workdir/src using target_dir / f.path. The .. path component moved the write into the parent directory, the build root. The user could therefore control both application source files and the configuration of the tool building them.
  2. Executing the configuration. The server ran Vite to build the React application. Vite loaded its configuration from the build root. Preserved configurations invoked Node.js child processes (child_process) and Python. Configuration code executed on the server with the build service process environment.
  3. Accessing Kubernetes. Kubernetes manages the containers running Ardor’s services. The build process used a service account named ardor-workload in main, the Kubernetes namespace grouping the production services. A service account is a program’s identity when calling the cluster API. Available permissions included reading Secrets objects containing service secrets and executing commands in other containers through pods/exec. This provided a path from one service to secrets and processes in neighboring services.
  4. Calling databases and providers. Obtained passwords and keys were used for direct SQL connections and API requests. Direct database access bypassed checks in Ardor's user-facing HTTP API: authorization then depended on the database role's permissions. An external provider key also allowed requests outside the platform.
  5. Moving from container to host. Separate actions used the Docker builder and privileged containers with access to host resources. These provided a path to root access on worker hosts. This transition is established separately from the earlier uid=0 output inside a container.

Insufficient path validation was the entry flaw; the process environment, service account permissions, and ability to launch privileged containers determined the reach of the attack. Broad permissions within individual Kubernetes namespaces did not amount to full cluster administration (cluster-admin): some requests for additional privileges were denied.

How results reached the attacker

The configuration could run commands and then call throw new Error(stdout + stderr) to put a child process's standard output and errors into a JavaScript exception. The service stored the build error and returned it to the client. Error messages became a channel for secrets and SQL results. An HTTP 201 response could still indicate successful creation of the application record, even though its build failed.

Preserved code and observed results must be distinguished. A read command in a file establishes intent; data in the returned error establishes that it reached the client. This analysis treats those returned results as evidence of disclosure, rather than treating command names without results as successful operations.

2. Timeline: two episodes on November 6 and 8

The preserved records distinguish two accounts: Ada Lovelace and John Tester. These are their display names in Ardor, not established real identities. Similar techniques and network indicators suggest a possible connection, but do not establish that one person controlled both accounts.

The email addresses were used to register the accounts; mailbox ownership and the operators’ real identities have not been established. The accounts used the same email domain and IP prefix; similar artifact names, user-agent strings, and techniques provide further indications of a possible connection. None of these establishes a shared operator. Network registration and IP geolocation describe infrastructure, not the attacker’s identity or physical location. The original geolocation fields are absent from the packaged Auth0 records, so the Tokyo label is retained only as summary context.

How the IP observations were attributed

The sources record addresses differently: forwarded request headers in the application logs, event IPs in Auth0, remote addresses in Stripe, and caller IPs in Google Cloud audit logs. Seven retained Cloudflare edge samples independently support the observed addresses. Matching a build request by time and path is still a correlation where no request-ID or request-body link is available. Ardor’s shared outbound address is a separate source and is not treated as an attacker’s external address.

Two additional Auth0-only profiles, ardorcm29zs79 and ardor4mgrw5r1, were created on November 6 at 12:33:34 and 12:34:34 UTC and had a recorded last IP of 64.34.94.217. Exploitation by those profiles has not been confirmed; they are not counted as additional confirmed attack accounts.

Names such as i49recon2 and rce214-8 are labels assigned to attacker-created artifacts, not Ardor service names. We retain them so individual attempts can be matched to the code when it is released.

All times below are UTC. Add eight hours for Singapore time. Times are shown to the second; clocks across systems were not independently calibrated. The tables show selected preserved events, not a complete recording of every action.

The following chart combines the pace of preserved build requests with selected observed stages on the same time axis. It shows the series of attempts behind the results. A bar counts build requests, not commands or successes; a dot marks an observed stage. Differences between timestamps from separate systems are not precise measurements of command execution duration.

Loading interactive chart…

Time is measured from each account’s registration. Bars use one-minute UTC bins and count server handler starts; stage dots use artifact record timestamps. Empty intervals mean no preserved build requests in this series, not proof of inactivity. Hover over or focus a bar or dot for its time and details. The detailed chronology below includes failed probes and changes in approach.

November 6: Ada Lovelace

The account profile was created at 12:35:15 UTC. The first preserved application version appeared at 12:46:29. Over approximately the next 21 minutes, 155 versions of 153 artifacts were preserved. The stages below include probes that had not yet produced confirmed execution. Later in the series, several lines of activity were interleaved; their order here does not mean that every action was a prerequisite for the next.

  1. 12:46:29–12:49:27

    The first seven probes

    The series began with the HTML artifact i49probe-html: its build was skipped, with no evidence of command execution. Six new artifacts followed: i49rce-probe, i49cfg-probe and i49rce2–i49rce5. Their preserved code includes commands such as id, reading /etc/passwd, listing directories, and checking system information.

    Build statuses differed: the first of those six failed, and the next five were marked successful. These statuses alone do not establish whether the embedded commands ran. This stage represents a search for an execution path, not six confirmed successful executions.

  2. 12:50:51–12:50:59

    Six configuration-placement variants

    A short series of cfgt-* artifacts followed. It tested configurations with the extensions .js, .mjs and .cjs in the build root, variants in the source directory, and another variant named art_js. This shows a specific search for a location from which the build tool would load a user-supplied file.

    UTCProbePreserved response
    12:50:51cfgt-root_jsError containing the marker CONFIGLOADED_root_js: the root configuration executed.
    12:50:53cfgt-root_mjsError containing the marker CONFIGLOADED_root_mjs: a second configuration variant was also loaded.
    12:50:54–12:50:59root_cjs, src_js, src_mjs, art_jsBuilds were marked successful; an equivalent execution marker has not been established here for these variants.

    The markers in the first two errors establish that the configuration became executable server-side code. In these probes, the error exposed the test result.

  3. 12:51:15–12:55:47

    From markers to commands and environment

    In i49recon1, the code already included checks of the user and system, reads of system files within the execution environment, and creation of a proof file. The build was marked successful, but that does not independently confirm each command's result. The next clear boundary was i49recon2 at 12:53:52: the build error preserved shell output, including uid=0(root) and information about the service environment.

    At 12:55:47, the response to i61-sa1 was already returning sensitive material from the service environment or private keys. The series had moved from checking execution to obtaining data available to the server process. The early uid=0 output concerns the container environment; host access is established by later actions.

  4. 12:57–12:59

    Service credentials and neighboring services

    Probes of Kubernetes, the environment, source code, and internal services were interleaved: i61-k8s*, i63app1, src*, i62recon* and others. Several preserved errors returned configuration and key material. At 12:58:35, i62secrets returned Kubernetes Secrets, PEM private keys, and external service credentials.

    This provided broader authority than the original user account: subsequent requests could use database passwords, Kubernetes service tokens, and provider keys. Each request's result must be assessed separately; possessing a key does not establish successful use of every API.

  5. 12:59:44–13:02:46

    Host-access and cloud API probes

    In i67-esc1, a privileged Docker container was created and launched through the internal image builder; the response preserved uid=0. Cloud and system probes continued. At 13:01:33, tokens for several Azure APIs were obtained, and at 13:02:46, i67-node3 read node files through a block device and debugfs, a tool for inspecting a filesystem directly. The result contained azure.json, a file holding Azure configuration for the node, and configuration for kubelet, the agent that manages containers on a Kubernetes node.

    Reading these files connects the series to the host environment. Cloud tokens do not establish control of the entire subscription: observed resource operations and potential permissions remain separate facts.

  6. 13:03:11–13:04:53

    Databases, the secret store, and a write test

    At 13:03:11, i62db3 directly read other users', administrators', and membership records. At 13:03:52, rec1 returned 2,767 workspace descriptions from the legacy accounting system, including 2,766 other workspaces.

    At 13:04:17, i62kdb2 examined the user secret store's structure: record metadata, encryption parameters, and the lengths of encrypted values were returned. Some encrypted records were read within the process for unsuccessful decryption attempts. Delivery of those records to the client and successful recovery of plaintext secrets have not been established.

    At 13:04:51, issuance of delegated Azure Storage authority and creation of a temporary access link were confirmed. At 13:04:53, i63blob1 tested writing an object: the result reported success, and the object's existence and checksum were independently checked. This establishes a write, beyond simply listing available objects.

  7. 13:05–13:06:08

    Accounting changes and further host probes

    At 13:05:18, the handler accepted a forged payment event, but downstream processing produced validation failures; a credit resulting from this attempt is not confirmed. Stripe's external log separately confirms creation of a trial subscription at 13:05:26. At 13:05:54, direct SQL changed balance and plan records, bypassing the user-facing payment API. This changed application data; it was not a real payment.

    Privileged-container tests continued in parallel: i61-c12 created a container, executed commands, and deleted it at 13:05:49; i61-c12b repeated the test with the host filesystem mounted at 13:06:08. Preserved results include reading azure.json, kubelet and SSH metadata, retrieving output, and deleting the container. Between these probes, at 13:05:57, reading another service's environment returned SECRETS_ENCRYPTION_KEY, a secret used for encryption.

  8. 13:06:24–14:18:57

    A background channel and the last preserved actions

    At 13:06:24, i63c2deploy wrote and launched a background program that read commands from a published file. Cloud storage version history preserved two output/state pairs linked to specific command versions. This was a separate way to continue execution outside the next build; its implementation is described in the persistence section.

    At 13:06:36, wake1 started another workspace's Git container by scaling it from zero instances to one. An SSH attempt timed out; wake2 confirmed command execution inside the container and process and port listings, but not repository reads. At 13:07:11, i61-c14c obtained a Google Cloud token and lists of project and storage metadata; other requests were denied or returned errors.

    The last preserved application version is timestamped 13:07:12, while periodic writes from the background program continued until 14:18:57. This shows activity beyond the build series, but does not establish the exact time access ended.

November 8: John Tester

The account registered at 05:22:47 UTC. Between the first preserved attempt at 05:27:13 and the last version at 05:40:52, 86 new artifacts were preserved. Environment reads, cloud probes, and persistence attempts were interleaved. In many cases, a build error contained the output of commands that had already run.

  1. 05:27:13–05:28:32

    Initial execution checks

    rce-check-pod contained commands to check the user, directories, and operating system. Three separate artifacts named A, B and C were then created within three seconds. All four builds were marked failed; this status does not establish the outcome of their individual operations.

    At 05:28:32, rce-cmd appeared with commands for id, directory listings, and reading /etc/passwd. The reviewed response suggests shell execution, but is not sufficient on its own to establish root access. Later steps provide separate evidence of host-level access.

  2. 05:28:36–05:30:58

    Environment, service code, and the service account

    In rce-cmd-2 at 05:28:36, environment and service account data were returned. The next probe, rce-cmd-3, targeted build pipeline code and publication routes. In rce-cmd-4–rce-cmd-6, the code accessed the Kubernetes service account token and certificate using curl and Python. These targets are visible in code; each command is not treated as an independently successful read.

    The rce2* series continued. A significant confirmed result was rce2-3 at 05:30:58: the build error returned the identifier and secret for Auth0’s Management API, used to administer the external authentication service that handles Ardor sign-ins. This preceded its subsequent use at the provider.

  3. 05:31:46–05:34:18

    Using obtained credentials and attempting user impersonation

    At 05:31:46, Auth0 recorded a client_credentials exchange, in which an application authenticates using its identifier and secret, from the episode's external address. It issued a token with 222 permission scopes. At 05:32:41, rce2-9 returned a wider set of service, database, model, and signing secrets. The same Auth0 secret appeared in both responses, confirming that it had been obtained before the token request.

    At 05:32:52, Google Cloud recorded token issuance on behalf of a service account. Then, at 05:33:39, a successful PATCH in Auth0 changed app_metadata.internal_id in the account's own profile to another user's identifier; a token was issued at 05:33:43, and the field was restored at 05:34:18. However, Ardor identifies the signed-in user through the authentication provider’s subject identifier (sub/auth0_id), rather than this editable metadata field. Successful operation as another user through this change has not been confirmed. Google Cloud service account use is a separate, confirmed mechanism.

  4. 05:34:55–05:36:26

    Other users' profiles, files, and a backup

    rce2-19 returned five profiles, four bcrypt password hashes, and three previously expired tokens at 05:34:55. Interleaved with subsequent rce2* probes was the azprobe-* series targeting cloud storage.

    The specific result of azprobe-13 at 05:36:16 was part of the globals.sql backup containing three PostgreSQL SCRAM password verifiers—stored authentication data, not plaintext passwords—and other workspaces' deployment descriptions. This confirms reading stored text. The CREATE/ALTER commands inside that file do not establish that the attacker executed them against the database.

  5. 05:36:30–05:37:11

    Direct writes bypass user-facing API checks

    At 05:36:30, rce2-25 performed a direct UPDATE: the account's global role changed from USER to ADMIN, and its membership role to Admin. Database records confirm these changes.

    After the intermediate probes rce2-26 and rce2-27, at 05:37:11, rce2-28 changed balance records and the assigned plan. For this request, Vite startup, the SQL write, the plan change, the build error, and the subsequent HTTP response 201 are linked. The error occurred after the data changed. Database role permissions enabled bypassing HTTP API checks.

  6. 05:37:15–05:38:36

    A command service and a separate series of host probes

    At 05:37:15, rce2-29 created configuration, a deployment, and an internal network service for an HTTP command shell. Three creation responses with status 201 and root output were preserved; the objects were found in the cluster. This is a confirmed execution mechanism, beyond a source file expressing intent to install one.

    A separate rce214* series had been running since 05:37:00. The initial artifact and variants -2–-7 were preserved before 05:38:36; several results contained environment values or key material. At 05:38:36, rce214-8 deleted sysdiag-x1 and created sysdiag-x2 with elevated container privileges (privileged), access to the host’s process and network namespaces (hostPID and hostNetwork), and the host filesystem mounted into it. Preserved results include reading /etc/shadow and a node root shell through nsenter, a tool for entering another process’s namespaces, with the marker HOSTSHELL_OK. The sequence shows removal of one object and creation of the next; it does not establish why the replacement occurred.

    Between these actions, at 05:37:59, azprobe-19 separately tested an Azure Blob write: successful object creation, retrieval of its properties, and return of the written test value were preserved. This confirms a write and a result check. It should not be conflated with a separate Google Cloud test object whose author is unknown.

  7. 05:39:19–05:39:49

    An overly broad role is denied; a revised attempt succeeds

    After rce214-9 and -10, at 05:39:19, rce214-11 created the service account system-maintenance and a token object. An attempt to add an overly broad role received Forbidden; the token itself had already been returned in the build error.

    The next artifact, rce214-12 at 05:39:32, created a role within the original permissions and bound it to the new service account. These objects were independently confirmed. At 05:39:49, rce214-13 created system-maintenance-agent and tested secret reads and command execution in containers using the new token. The deployment and container remained running. This shows adaptation after a denial: the first role failed, while the more limited role succeeded.

  8. 05:40:02–05:41:28

    Checking persistence, neighboring services, and external reads

    At 05:40:02, rce214-14 ran root commands on the host through nsenter, wrote a proof file, and read azure.json and the kubelet bootstrap configuration. The write result is visible in the output; the file was later independently found on the preserved node copy.

    At 05:40:12, rce214-15 executed commands in two other services: the service managing user application deployments and the service managing user runtime environments. Responses contained database configuration, internal API keys, signing secrets, and part of a service token. At 05:40:20, rce214-16 read the created token again and successfully called the service account API with it. The recorded token fields did not include exp, the expiration timestamp.

    The last version, rce214-17, was preserved at 05:40:52. Stripe then processed 15 pages of the general customer list from the episode's external address at 05:41:20–05:41:28. The build series therefore ended before the last confirmed external activity.

How the attempts changed

The sequence of code and responses shows revisions: configuration placement changed; an overly broad role denial was followed by a more limited role and binding; decryption failures were followed by checks of other secret sources; malformed Stripe filters were replaced with working requests. This establishes adaptation to system responses. Request speed and shape do not establish use of AI, the number of authors, or the degree of automation.

The first episode contains 155 preserved versions over approximately 21 minutes; the second contains 86 over approximately 14 minutes. These are application versions, each potentially containing several operations, rather than a count of commands.

3. Data and access affected

The consequences extended beyond the accounts used for the attack. The recorded build responses contained data concerning at least 37 other users. This is a confirmed minimum, not a complete count of everyone affected. This included email addresses, membership and billing information, and some authentication data. The set includes five complete profiles, four password hashes, and three tokens that had expired in 2025. A password hash is not a plaintext password; possession of an expired token does not establish the ability to sign in with it.

The legacy accounting system returned metadata for 2,766 other workspaces, the objects grouping user applications and settings. Disclosure of individual files, deployment descriptions, Git configuration, or runtime fragments is confirmed in 14 workspaces. These counts describe different, potentially overlapping types of disclosure and cannot be added into a total number of affected users.

Available credentials potentially covered 12 application databases. Successful connections are confirmed in eight; some results contain only table lists and schema information, while others include application records. Reading every table or exporting all 12 databases has not been established.

Secrets belonging to users and internal services

Here, “secrets” means API keys, service passwords, private keys, and other credentials allowing action on behalf of an application or account. Their disclosure can enable access beyond Ardor.

Build responses returned service secrets, including an encryption key used by one Ardor service. The exact mapping of that key to every record in the user secret store has not been established. Preserved attempts to decrypt individual records failed; successful export of plaintext user secrets is not confirmed by these materials. Those failures also do not establish that other records were protected.

For incident response, potentially accessible secrets were treated as compromised. That determined the need for revocation and replacement, but does not mean that extraction of every secret is proven. The full volume of data obtained and any subsequent redistribution remain unknown.

A complete history of database and Kubernetes requests is unavailable. The confirmed disclosures described here therefore do not establish the full extent of access or rule out additional reads.

4. Activity in external services

Stripe: confirmed use of a key

Stripe logs confirmed 36 external requests from two addresses associated with the episodes. In the second episode, 15 pages of the general customer list were retrieved in approximately eight seconds. The available logs do not contain the response bodies: page count and response size cannot establish the exact customer count or returned fields.

These requests confirm that the compromised key was used to access Stripe, the payment provider. Export of full card numbers and CVC has not been established: the examined API methods do not return those values. Individual events may have contained masked card information, but its contents are unknown without response bodies.

Separately, 838 requests through Ardor's shared server egress address were examined in two time windows. Application and attacker requests behind that address cannot be distinguished by IP alone. Most concern data associated with the two accounts, which does not establish which process sent them. The source of 17 requests remains unknown. All 838 cannot be treated as distinct attacker actions.

Cloud infrastructure and authentication

Azure object reads and writes and limited infrastructure metadata requests are confirmed. Available permissions exceeded the observed operations, but takeover of the entire subscription, publication of container images, and reading cloud vault secret values have not been established.

In Google Cloud, four tokens were issued on behalf of service accounts from the second episode's address. Resource and storage object listing requests are also preserved. Listing objects does not establish downloading their contents. Reads of object bodies and Google Cloud secret values are not confirmed by the available records.

In Auth0, obtaining a management token and modifying the account's own data are confirmed. Attempting to associate it with another user's internal identifier is not a proven login as that user. This must be distinguished from confirmed actions on behalf of Google Cloud service accounts.

The available Auth0 logs do not cover November 6, so they cannot establish the full authentication activity for the first episode.

Entry into Ardor's separate test environment has not been established. Matching network indicators and potential paths through shared credentials warrant investigation, but do not prove successful intrusion.

5. Attempts to maintain access

November 6: commands through a published file

The background program i63c2d.py checked the published cmd.html every three seconds. A changing timestamp was appended to the URL to avoid cached commands. When the text differed from the previous command, the program ran it through /bin/bash -lc. Execution had a 900-second timeout.

Output was capped at 6 MiB and written to out.txt, with state in state.txt. These files were overwritten, but cloud storage version history preserved two corresponding output/state pairs. Command checksums matched preserved versions of cmd.html; both pairs contained exit code zero and a root indicator. This links commands to execution, beyond showing that the program existed on disk.

The first confirmed command printed a proof marker and the user identity. The second printed the hostname, user identity, a listing of /app, the existence and sizes of environment files, and names of files related to providers. Finding a filename alone does not establish reading its contents. The code also supported file transfers, but their presence does not confirm actual use.

The background channel outlasted the build series

Heartbeat writes continued for about 72 minutes after the last preserved application version. November 6, 2026 · UTC.

  1. Background command channel launched

    The program polled a published file for commands, allowing execution outside subsequent builds.

  2. Last preserved application version

    The saved build series ended; the background program continued writing activity signals.

  3. Last preserved heartbeat write

    About 72 minutes after the last version. This is the end of the retained heartbeat series, not an established time at which access stopped.

137 heartbeat records; two confirmed executed commands. Heartbeats indicate that the background program was active. The two commands are confirmed separately by matching output/state pairs and command checksums; the heartbeat count is not a command count.

Events are arranged in sequence; spacing is not proportional to elapsed time. The underlying heartbeat timestamps and intervals are available in the expandable data table.

November 8: an HTTP command service and host access

At 05:37:15, the account created a ConfigMap containing the program, a Deployment to run it, and a Service of type ClusterIP. These are, respectively, configuration, a container deployment definition, and an internal Kubernetes network service. The Python HTTP server listened on port 8765; the service forwarded port 80 to it. Commands received through GET requests were passed to subprocess with shell=True and a 25-second timeout.

Object creation and root output are confirmed. However, ClusterIP is an internal address type, and the preserved records do not establish Internet exposure of the service. They also do not establish that all later commands used it.

At 05:38:36, the container sysdiag-x2 was created with elevated privileges and access to the host’s process and network namespaces, and the host filesystem was mounted at /host with write access. This provided access to host resources and processes. Root command output through nsenter, a tool for entering host process namespaces, was preserved. A marker file written to the host at 05:40:02 was later independently found on the preserved host copy.

A new service account after excessive permissions were denied

At 05:39:19, the service account system-maintenance and an object for its token were created. The first attempt to assign a broad role received Forbidden. At 05:39:32, a more limited Kubernetes permission set (Role) and an assignment (RoleBinding) granting it to the service account were created. At 05:39:49, successful secret reads, container listings, and command execution using the new token were preserved.

The recorded token fields did not include an expiration timestamp, exp. This was a separate access mechanism created during the attack, so disabling the original user account alone would not establish its termination. Denial of the first role also did not block the entire persistence attempt: subsequent operations with the limited role are confirmed.

These traces establish specific persistence mechanisms. They do not establish Internet exposure of the command service or use of every data transfer capability present in the code. Some actions may have left no result in the available history.

6. Detection, recovery, and changes

We detected the incident on November 8 and immediately switched the system into emergency mode: we disabled user applications and blocked their connections to the outside world. This was necessary to stop untrusted code execution and prevent use of compromised secrets through the affected infrastructure.

We then separately restricted service account permissions, revoked and replaced secrets, stopped affected hosts, and preserved investigation material. Stopping an application stops its process, but does not revoke an external service key that has already been copied.

November 8 · Response timeline (UTC)

  1. Around 10:05

    Selected Azure service account permissions were revoked, including access to storage, the image registry, and the key vault.

  2. 11:17:25

    The journal recorded shutdown of user compute and suspension of schedules. Data on persistent disks was retained.

  3. Around 12:15

    Two affected hosts were stopped and quarantined.

  4. 12:20:11

    A check of the old Cloudflare DNS management credential returned an authorization failure.

The timeline lists selected actions with recorded timestamps. The available response records do not give the exact detection time or first switch to emergency mode. The later complete shutdown of user compute should not be read as the time we first detected or restricted the incident.

Response records also confirm disabling new logins for 11 database roles, terminating 53 connections, and rejection of the old GitHub App key. Known persistence mechanisms were removed. Disk snapshots and two memory images were preserved for further investigation. Preserving images does not mean their contents have already been fully analyzed.

Checks of 13 running hosts found no matches for the attack indicators used. This is the result of specific checks, rather than a complete reconstruction of every host's history. Kubernetes audit logging was not configured, limiting reconstruction of earlier actions.

What the investigation changed

The attack exposed three connected failures: untrusted build configuration ran inside a process holding service credentials; that process had excessive access to other services; and privileged workloads provided a route to the underlying hosts. Recovery therefore required changes to execution, permissions, and infrastructure, as well as credential replacement.

Rebuilding production

We provisioned a new Kubernetes cluster and moved production services to it. The compromised environment was taken out of production service. The old application hosts, operations host, and database server were stopped; investigation material was retained separately. We did not return the compromised hosts to production.

We disabled the attacker accounts, revoked their unauthorized administrative access, and removed the identified persistence mechanisms: attacker-created service accounts and tokens, command services, and background processes. The attacker accounts’ active resources were removed from the production environment.

Closing the build-service entry point

Web-page builds now run in separate isolated environments (Sandboxes) instead of inside the API process that holds service credentials. The new build path validates file paths and resource limits, does not give those Sandboxes an identity for accessing cloud resources, restricts their network access, and verifies that signed completion messages belong to the expected job and have not expired. This removes the original route from user-supplied Vite configuration into the credential-bearing API process.

We also removed service secrets from the separate container image builder and restricted access to it. Other application build and deployment paths remain temporarily restricted while we complete additional security checks for similar attack chains.

Reducing access between services

In the rebuilt environment, we replaced the shared service-access model with separate service accounts and cloud identities, scoped to the resources each service needs. The remaining cloud identity used by the application deployment service has restricted permissions for that service, rather than broad access across the storage account or permission to push images to the platform registry. The temporary identity used for migration was removed after the transfer.

We added admission rules, which check requested changes before Kubernetes accepts them. For the controller that creates and manages user environments, these rules reject changes outside the permitted namespaces, privileged containers, use of host process or network namespaces, and unapproved host filesystem mounts. This adds a check between an application request and the resources it can create.

Replacing credentials and checking the result

We rotated all platform service credentials, including database passwords, API keys, service tokens, and email-provider keys, and updated the services that use them. We also terminated existing database sessions, refreshed credentials in user runtime environments, and verified that previous credentials were rejected in the rotation checks.

We replaced the exposed encryption key and rotated the associated secrets under Ardor’s control. We notified all users that external-service credentials must be revoked or replaced at the services that issued them. Ardor’s platform rotation does not replace that customer-side step.

Restoring service and adding visibility

In the November 15 production check, the public console responded successfully, all 11 cluster nodes were ready, and all 289 existing pods had ready containers. This confirmed infrastructure readiness at that time; it was not a test of every customer workflow.

The rebuilt cluster now sends Kubernetes administrative audit events and security events to an external logging workspace. This adds visibility that was missing during the incident.

7. Further investigation and code release

We are preparing a separate release of the attacker code and supporting technical material so readers can examine the individual attempts behind this account. We will link the relevant examples from the attack timeline when that material is published.

The lessons are already clear: user-controlled files must not become trusted build configuration inside a service holding platform credentials. Separating execution, limiting service permissions, and restricting privileged workloads address different steps of the attack chain.