Teleport 14 Test Plan #31122
Labels
test-plan
A list of tasks required to ship a successful product release.
Comments
r0mant
added
the
test-plan
Test plan cont'd (due to Github's issue description size limit).Desktop Access @ibeckermayer @probakowski
Binaries compatibility @fheinecke
Machine ID @strideynetSSHWith a default Teleport instance configured with a SSH node:
Ensure the above tests are completed for both:
DB AccessWith a default Postgres DB instance, a Teleport instance configured with DB access and a bot user configured:
Host users creation @jakuleHost users creation docs
CA rotations @espadolini
Proxy Peering
EC2 Discovery @marcoandredinis
Azure Discovery @hugoShaka
GCP Discovery @tcsc
IP Pinning @AntonAMAdd a role with
Resources |
|
Edit: fixed |
PostgreSQL 10k testSetupAzure Database for PostgreSQL Flexible Server 15.3 on AKS with Kubernetes 1.26.6, 15 nodes Teleport configured with 3 auths and 3 proxies, PostgreSQL 10k tunnel nodesMetrics
Soak testRan from a pod in the same cluster as the control plane and the nodes. 5k sessions (on 1k tunnel nodes) then 10k direct connect nodesMetrics
Soak testRan from a pod in the same cluster as the control plane and the nodes. Concurrent sessions testNo errors reported by 
This is not immediate cause for concern, as real workloads will never shut down 5000 sessions at the exact same time, but it might be possible to improve the behavior with some tuning (reducing the size of the connection pool might help, reducing the actual need for retries due to contention). |
|
In addition to test plan tasks, Machine ID was also test for Kubernetes Access and Application Access ( I will add these to test plan before T15 ) |
This was referenced Aug 31, 2023
|
Edit: solved. |
|
Loading all CAs for |
|
|
|
I scratched off items for "Test Databases screen in the web UI" since that screen was removed and replaced by the unified resource view. I did verify that searching, filtering, sorting etc work in the unified resources view for databases however. Need to update those testplan steps @avatus |
|
Will do, thanks @GavinFrazar . #31214 |
|
|
Firestore 10k testMetrics
Soak testSingle node, random and label-based (single node) 30min soak tests passed with no errors. |
|
AWS roles do not show up when trying to log into the AWS console in the UI in the new unified resource view: #31573 |
|
Azure Discovery keeps running the discovery script on already-joined VMs every 10 minutes, but it seems the bug was here in 13: #28879 |
|
Azure Discovery permissions are not up-to-date and following the docs doesn't allow to setup a working discovery service: #31602 |
|
Agents running versions older than v14 are not able to connect to a v14 cluster: #31607 |
Cloud Load Tests30k Scaling Test
https://grafana-staging-onprem.platform.teleport.sh/goto/60SnOjzIR?orgId=1 10k Concurrent Sessions
Soak TestsOrigin: us-east-1 Target: us-east-1kubectl logs -n soaktest -f pod/soaktest-7zpdz-56wqw
+ tsh --proxy=benchmark.cloud.gravitational.io:443 -i /etc/teleport/auth bench ssh --duration=30m root@node-agents-766996b7b9-zv7b2-09 ls
* Requests originated: 17999
* Requests failed: 0
Histogram
Percentile Response Duration
---------- -----------------
25 191 ms
50 195 ms
75 201 ms
90 210 ms
95 216 ms
99 270 ms
100 5199 ms
+ tsh --proxy=benchmark.cloud.gravitational.io:443 -i /etc/teleport/auth bench ssh --duration=30m root@fullname=node-agents-766996b7b9-zv7b2-09 ls
* Requests originated: 17996
* Requests failed: 0
Histogram
Percentile Response Duration
---------- -----------------
25 466 ms
50 473 ms
75 481 ms
90 493 ms
95 505 ms
99 551 ms
100 1193 ms
+ tsh --proxy=benchmark.cloud.gravitational.io:443 -i /etc/teleport/auth bench ssh --duration=30m --random root@all ls
* Requests originated: 17999
* Requests failed: 0
Histogram
Percentile Response Duration
---------- -----------------
25 188 ms
50 195 ms
75 203 ms
90 215 ms
95 229 ms
99 284 ms
100 9191 mshttps://grafana-staging-onprem.platform.teleport.sh/goto/QDvmOCzIR?orgId=1 Origin: us-west-2 Target: us-east-1kubectl logs -n soaktest -f pod/soaktest-rkbzk-zvf2z
+ tbot start --data-dir=/var/lib/teleport/bot --destination-dir=/opt/machine-id --token=163cbdb82281e399049c8034ef77219b --join-method=token --auth-server=benchmark.cloud.gravitational.io:443 --certificate-ttl=8h --oneshot
[TBOT] INFO Anonymous telemetry is not enabled. Find out more about Machine ID's anonymous telemetry at https://goteleport.com/docs/machine-id/reference/telemetry/ tbot/anonymous_telemetry.go:82
[TBOT] INFO Created directory "/var/lib/teleport/bot" config/destination_directory.go:135
[TBOT] INFO Created directory "/opt/machine-id" config/destination_directory.go:135
[TBOT] INFO Initializing bot identity. tbot/tbot.go:254
[TBOT] INFO Loading existing bot identity from store. store:directory: /var/lib/teleport/bot tbot/tbot.go:325
[TBOT] INFO No existing bot identity found in store. Bot will join using configured token. tbot/tbot.go:329
[TBOT] INFO Fetching bot identity using token. tbot/bot_identity.go:193
[AUTH] INFO Attempting registration via proxy server. auth/register.go:278
[AUTH] INFO Successfully registered via proxy server. auth/register.go:285
[TBOT] INFO Fetched new bot identity. identity:valid: after=2023-09-05T19:16:47Z, before=2023-09-06T03:17:47Z, duration=8h1m0s | kind=tls, renewable=true, disallow-reissue=false, roles=[bot-soaktest-bot], principals=[-teleport-internal-join], generation=1 tbot/tbot.go:298
[TBOT] INFO Bot initialization complete. tbot/tbot.go:316
[TBOT] INFO One-shot mode enabled. Generating outputs. tbot/tbot.go:118
[TBOT] INFO Generating output. output:identity (directory: /opt/machine-id) tbot/impersonated_identity.go:528
[TBOT] INFO Generated output. output:identity (directory: /opt/machine-id) tbot/impersonated_identity.go:573
[TBOT] INFO Generated outputs. One-shot mode is enabled so exiting. tbot/tbot.go:123
+ tsh --proxy=benchmark.cloud.gravitational.io:443 -i /opt/machine-id/identity bench ssh --duration=30m root@node-agents-5d68d45658-25b9q-00 ls
* Requests originated: 17992
* Requests failed: 0
Histogram
Percentile Response Duration
---------- -----------------
25 881 ms
50 892 ms
75 912 ms
90 930 ms
95 936 ms
99 951 ms
100 5619 ms
+ tsh --proxy=benchmark.cloud.gravitational.io:443 -i /opt/machine-id/identity bench ssh --duration=30m root@fullname=node-agents-5d68d45658-25b9q-00 ls
* Requests originated: 17991
* Requests failed: 0
Histogram
Percentile Response Duration
---------- -----------------
25 903 ms
50 917 ms
75 948 ms
90 1027 ms
95 1039 ms
99 1054 ms
100 1136 mshttps://grafana-staging-onprem.platform.teleport.sh/goto/S80kdCzIR?orgId=1 Origin: us-east-1 Target: us-west-2kubectl logs -n soaktest -f pod/soaktest-9vg9b-k458s
+ tbot start --data-dir=/var/lib/teleport/bot --destination-dir=/opt/machine-id --token=b29218b11195ef04f77b1ab93b7382fc --join-method=token --auth-server=benchmark.cloud.gravitational.io:443 --certificate-ttl=8h --oneshot
INFO [TBOT] Created directory "/var/lib/teleport/bot" config/destination_directory.go:135
INFO [TBOT] Anonymous telemetry is not enabled. Find out more about Machine ID's anonymous telemetry at https://goteleport.com/docs/machine-id/reference/telemetry/ tbot/anonymous_telemetry.go:82
INFO [TBOT] Created directory "/opt/machine-id" config/destination_directory.go:135
INFO [TBOT] Initializing bot identity. tbot/tbot.go:254
INFO [TBOT] Loading existing bot identity from store. store:directory: /var/lib/teleport/bot tbot/tbot.go:325
INFO [TBOT] No existing bot identity found in store. Bot will join using configured token. tbot/tbot.go:329
INFO [TBOT] Fetching bot identity using token. tbot/bot_identity.go:193
INFO [AUTH] Attempting registration via proxy server. auth/register.go:278
INFO [AUTH] Successfully registered via proxy server. auth/register.go:285
INFO [TBOT] Fetched new bot identity. identity:valid: after=2023-09-05T20:25:33Z, before=2023-09-06T04:26:32Z, duration=8h0m59s | kind=tls, renewable=true, disallow-reissue=false, roles=[bot-soaktest-bot], principals=[-teleport-internal-join], generation=1 tbot/tbot.go:298
INFO [TBOT] Bot initialization complete. tbot/tbot.go:316
INFO [TBOT] One-shot mode enabled. Generating outputs. tbot/tbot.go:118
INFO [TBOT] Generating output. output:identity (directory: /opt/machine-id) tbot/impersonated_identity.go:528
INFO [TBOT] Generated output. output:identity (directory: /opt/machine-id) tbot/impersonated_identity.go:573
INFO [TBOT] Generated outputs. One-shot mode is enabled so exiting. tbot/tbot.go:123
+ tsh --proxy=benchmark.cloud.gravitational.io:443 -i /opt/machine-id/identity bench ssh --duration=30m root@node-agents-5d68d45658-z8nz7-00 ls
* Requests originated: 17992
* Requests failed: 0
Histogram
Percentile Response Duration
---------- -----------------
25 854 ms
50 863 ms
75 868 ms
90 875 ms
95 880 ms
99 910 ms
100 1270 ms
+ tsh --proxy=benchmark.cloud.gravitational.io:443 -i /opt/machine-id/identity bench ssh --duration=30m root@fullname=node-agents-5d68d45658-z8nz7-00 ls
* Requests originated: 17989
* Requests failed: 0
Histogram
Percentile Response Duration
---------- -----------------
25 1125 ms
50 1133 ms
75 1143 ms
90 1151 ms
95 1159 ms
99 1191 ms
100 3631 mshttps://grafana-staging-onprem.platform.teleport.sh/goto/S80kdCzIR?orgId=1 |
ETCD 10k Loadtest (simulated)This is a first attempt at creating a "simulated" loadtesting procedure. The procedure is simulated in that it stresses the backend by using For this initial attempt the following command was run concurrently on each auth pod (note the 5k count, totaling 10k since two auth pods were created): tctl loadtest node-heartbeats --count=5000 --ttl=2m --interval=1m --labels=2 --concurrency=32This method of loadtesting produces significantly less load on the control-plane generally, but roughly equivalent load on the backend itself: 
In addition to the above metrics, auth logs were specifically monitored for any cache and/or event system related errors. While we don't anticipate such things at 10k these days, such errors would be one of the main signs of regression that might not be immediately obvious. As a point of future improvement, I think we should start tracking cache resets and watcher buffer overflows as part of our standard suite of backend metrics so that we can better monitor the health of the event system. |
ETCD 30k Loadtest (simulated)See #31122 (comment) for explanation of simulated loadtest procedure. The 30K procedure was identical, except that 15k heartbeats were applied per auth-server instead of 5k. 
As with the 10k procedure, logs were explicitly monitored for cache and event system issues. None were observed, but the metrics improvement thoughts from that comment still stand. |
Database Access load test (PostgreSQL and MySQL)SetupEKS with a single node group:
Teleport cluster (all deployed on the EKS cluster):
Databases:
Note: Databases were configured using discovery running inside the database agent.
MySQL10 connections/second
50 connections/second
NotesThe failed connection happened at the end of the benchmark test, where the final connections didn't have a chance to complete as PostgreSQL10 connections/second
50 connections/second
NotesMost of the connection failures were due to the proxy not being able to communicate with the database agent. LogsDuring the entire test the database agent was logging a warning of Worth noting that during the tests a single Audit instance was handling all the audit events, which could cause the delayed processing:
|
|
|
Kubernetes Access load testSetupEKS with a single node group:
Teleport cluster (all deployed on the EKS cluster):
|
|
|
|
Not sure if this is a bug, Azure VMs belonging to Scale Sets are not discovered: #31758 |
500 Trusted Clusters (etcd)
|
|
Playing a leaf SSH session recorded at the proxy fails: #31776 |
|
GCP Discovery appears totally broken. Existing issue: #31386 |
|
IP pinned users can't upload/download files after connecting to nodes in web UI #31845 |
ETCD Soak Testsnote: missing node during |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
and others
Manual Testing Plan
Below are the items that should be manually tested with each release of Teleport.
These tests should be run on both a fresh installation of the version to be released
as well as an upgrade of the previous version of Teleport.
Adding nodes to a cluster @bl-nero
Labels @bl-nero
Trusted Clusters @bl-nero
RBAC @bl-nero
Make sure that invalid and valid attempts are reflected in audit log. Do this with both Teleport and Agentless nodes.
Verify that custom PAM environment variables are available as expected. @atburke
Users @bl-nero
With every user combination, try to login and signup with invalid second
factor, invalid password to see how the system reacts.
WebAuthn in the release
tshbinary is implemented using libfido2 forlinux/macOS. Ask for a statically built pre-release binary for realistic
tests. (
tsh fido2 diagshould work in our binary.) Webauthn in Windowsbuild is implemented using
webauthn.dll. (tsh webauthn diagwithsecurity key selected in dialog should work.)
Touch ID requires a signed
tsh, ask for a signed pre-release binary so youmay run the tests.
Windows Webauthn requires Windows 10 19H1 and device capable of Windows
Hello.
Adding Users Password Only
Adding Users OTP
Adding Users WebAuthn
Adding Users via platform authenticator
Managing MFA devices
tsh mfa addtsh mfa addtsh mfa addtsh mfa lstsh mfa rmtsh mfa rmsecond_factor: oninauth_service, should failsecond_factor: optionalinauth_service, should succeedLogin Password Only
Login with MFA
tsh mfa addLogin OIDC
Login SAML
Login GitHub
Deleting Users
Backends @capnspacehook
Session Recording @capnspacehook
Enhanced Session Recording @jakule
disk,commandandnetworkevents are being logged.enhanced_recordingrole option.Restricted Session @jakule
Auditd @jakule
teleport/lib/auditd/common.go
Lines 25 to 34 in 7744f72
Audit Log @Joerger
Audit log with dynamodb
Audit log with Firestore
Failed login attempts are recorded
Interactive sessions have the correct Server ID
server_idis the ID of the node in "session_recording: node" modeserver_idis the ID of the node in "session_recording: proxy" modeforwarded_byis the ID of the proxy in "session_recording: proxy" modeNode/Proxy ID may be found at
/var/lib/teleport/host_uuidin thecorresponding machine.
Node IDs may also be queried via
tctl nodes ls.Exec commands are recorded
scpcommands are recordedSubsystem results are recorded
Subsystem testing may be achieved using both
Recording Proxy mode
and
OpenSSH integration.
Assuming the proxy is
proxy.example.com:3023andnode1is a node runningOpenSSH/sshd, you may use the following command to trigger a subsystem audit
log:
sftp -o "ProxyCommand ssh -o 'ForwardAgent yes' -p 3023 %[email protected] -s proxy:%h:%p" root@node1Interact with a cluster using
tsh@lxeaThese commands should ideally be tested for recording and non-recording modes as they are implemented in a different ways.
Interact with a cluster using
ssh@capnspacehookMake sure to test both recording and regular proxy modes.
Verify proxy jump functionality @Joerger
Log into leaf cluster via root, shut down the root proxy and verify proxy jump works.
Interact with a cluster using the Web UI @bl-nero
tshCA loading @atburkeCreate a trusted cluster pair with a node in the leaf cluster. Log into the root cluster.
load_all_cason the root auth server isfalse(default) -tsh ssh leaf.node.example.comresults in access denied.load_all_cason the root auth server istrue-tsh ssh leaf.node.example.comsucceeds.
X11 Forwarding @Joerger
xeyesandxclip:apt install x11-apps xclipxeyes. Thenbrew install xclip.ssh_service.x11.enabled = yestsh ssh -X user@node xeyestsh ssh -X root@node xeyestsh ssh -Y server01 "echo Hello World | xclip -sel c && xclip -sel c -o"should print "Hello World"tsh ssh -X server01 "echo Hello World | xclip -sel c && xclip -sel c -o"should fail with "BadAccess" X errorUser accounting @atburke
/var/run/utmpon Linux./var/log/wtmpon Linux.Combinations @marcoandredinis
For some manual testing, many combinations need to be tested. For example, for
interactive sessions the 12 combinations are below.
sshas client #31281Teleport with EKS/GKE @tigrato
Teleport with multiple Kubernetes clusters @AntonAM
Note: you can use GKE or EKS or minikube to run Kubernetes clusters.
Minikube is the only caveat - it's not reachable publicly so don't run a proxy there.
tsh login, check thattsh kube lshas your clusterkubectl get nodes,kubectl exec -it $SOME_POD -- shtsh login, check thattsh kube lshas your clusterkubectl get nodes,kubectl exec -it $SOME_POD -- shtsh login, check thattsh kube lshas your clusterkubectl get nodes,kubectl exec -it $SOME_POD -- shtsh login, check thattsh kube lshas both clusterstsh kube loginkubectl get nodes,kubectl exec -it $SOME_POD -- shon the new clustertsh login, check thattsh kube lshas all clustersnameandlabelsStep 2login value matching the rowsnamecolumnnameorlabelsin the search bar worksnamecolumKubernetes auto-discovery @tigrato
tctl create.tctl create -f.tctl rm.Kubernetes Secret Storage @AntonAM
StatefulsetKubernetes RBAC @AntonAM
kubernetes_resources:{"kind":"pod","name":"*","namespace":"*"}- must allow access to every pod.{"kind":"pod","name":"<somename>","namespace":"*"}- must allow access to pod<somename>in every namespace.{"kind":"pod","name":"*","namespace":"<somenamespace>"}- must allow access to any pod in<somenamespace>namespace.*wildcards -<some-name>-*and regex fornameandnamespacefields.go-client.kubernetes_resources:kubernetes_groupsthat denies exec into a podsearch_as_rolesis not allowed.kind: namespacescenarios forkubernetes_resources:kubernetes_groupsthat denies exec into a pod from another namespacekubernetes_resourcesis capable of restricting verbs:readand try to list, update and delete some resourceTeleport with FIPS mode @codingllama
ACME @bl-nero
Migrations @bl-nero
SSH should work for both main and old clusters
SSH should work
Command Templates
When interacting with a cluster, the following command templates are useful:
OpenSSH
Teleport
Teleport with SSO Providers
GitHub External SSO @capnspacehook
tctl ssofamily of commands @tcscFor help with setting up sso connectors, check out the [Quick GitHub/SAML/OIDC Setup Tips]
tctl sso configurehelps to construct a valid connector definition:tctl sso configure github ...creates valid connector definitionstctl sso configure oidc ...creates valid connector definitionstctl sso configure saml ...creates valid connector definitionstctl sso testtest a provided connector definition, which can be loaded fromfile or piped in with
tctl sso configureortctl get --with-secrets. Validconnectors are accepted, invalid are rejected with sensible error messages.
tctl sso test.Teleport Plugins @EdwardDowling
AWS Node Joining @atburke
Docs
ec2:DescribeInstancespermissions for local account:TELEPORT_TEST_EC2=1 go test ./integration -run TestEC2NodeJoinTELEPORT_TEST_EC2=1 go test ./integration -run TestIAMNodeJoinKubernetes Node Joining @hugoShaka
Azure Node Joining @tcsc
Docs
GCP Node Joining @tcsc
Docs
Cloud Labels @tcsc
and with tag
foo:bar. Verify that a node running on the instance has labelaws/foo=bar.foo:bar. Verify that a node running on theinstance has label
azure/foo=bar.Passwordless @codingllama
This feature has additional build requirements, so it should be tested with a pre-release build from Drone (eg:
https://get.gravitational.com/teleport-v10.0.0-alpha.2-linux-amd64-bin.tar.gz).This sections complements "Users -> Managing MFA devices".
tshbinaries foreach operating system (Linux, macOS and Windows) must be tested separately for
FIDO2 items.
Diagnostics
Commands should pass all tests.
tsh fido2 diag(macOS/Linux)tsh touchid diag(macOS only)tsh webauthnwin diag(Windows only)Registration
tsh mfa add, choose WEBAUTHN andpasswordless)
tsh mfa add, choose TOUCHID)tsh mfa add, choose WEBAUTHN andpasswordless)
Login
tsh login --auth=passwordless)tsh login --auth=passwordless)tsh login --auth=passwordless --mfa-mode=cross-platformuses FIDO2tsh login --auth=passwordless --mfa-mode=platformuses platform authenticatortsh login --auth=passwordless --mfa-mode=autoprefers platform authenticatorthe same device)
(
auth_service.authentication.passwordless = false)(
auth_service.authentication.connector_name = passwordless)(
tsh login --auth=local)Touch ID support commands
tsh touchid lsworkstsh touchid rmworks (careful, may lock you out!)Device Trust @codingllama
Device Trust requires Teleport Enterprise.
This feature has additional build requirements, so it should be tested with a
pre-release build from Drone (eg:
https://get.gravitational.com/teleport-v10.0.0-alpha.2-linux-amd64-bin.tar.gz).Client-side enrollment requires a signed
tshfor macOS, make sure to use thetshbinary fromtsh.app.A simple formula for testing device authorization is:
Inventory management
tctl devices add)tctl devices add --enroll)tctl devices ls)tctl devices rm)tctl devices rm)tctl devices enroll)tctl devices enroll)Device enrollment
Enroll device on macOS (
tsh device enroll)Enroll device on Windows (
tsh device enroll)Verify device extensions on TLS certificate
Note that different accesses have different certificates (Database, Kube,
etc).
Verify device extensions on SSH certificate
Device authorization
device_trust.mode other than "off" or "" not allowed (OSS)
device_trust.mode="off" doesn't impede access (Enterprise and OSS)
device_trust.mode="optional" doesn't impede access, but issues device
extensions on login
device_trust.mode="required" enforces enrolled devices
device_trust.mode="required" is enforced by processes, and not only by
Auth APIs
Testing this requires issuing a certificate without device extensions
(mode="off"), then changing the cluster configuration to mode="required" and
attempting to access a process directly, without a login attempt.
Role-based authz enforces enrolled devices
(device_trust.mode="off" or "optional",
role.spec.options.device_trust_mode="required")
Device authorization works correctly for both require_session_mfa=false
and require_session_mfa=true
Device authorization applies to SSH access (all items above)
Device authorization applies to Trusted Clusters (root with
mode="optional" and leaf with mode="required")
Device authorization applies to Database access (all items above)
Device authorization applies to Kubernetes access (all items above)
Device authorization does not apply to App access
(both cluster-wide and role)
Device authorization does not apply to Windows Desktop access
(both cluster-wide and role) (@ibeckermayer)
Device audit (see lib/events/codes.go)
data (for certificates with device extensions)
Binary support
tshfor macOS gives a sane errormessage for
tsh device enrollattempts.Device support commands
tsh device collect(macOS)tsh device asset-tag(macOS)tsh device collect(Windows)tsh device asset-tag(Windows)Hardware Key Support @jakule
Hardware Key Support is an Enterprise feature and is not available for OSS.
You will need a YubiKey 4.3+ to test this feature.
This feature has additional build requirements, so it should be tested with a pre-release build from Drone (eg:
https://get.gravitational.com/teleport-ent-v11.0.0-alpha.2-linux-amd64-bin.tar.gz).Server Access
These tests should be carried out sequentially.
tshtests should be carried out on Linux, MacOS, and Windows.tsh loginas user with Webauthn login and no hardware key requirement.role.role_options.require_session_mfa: hardware_key-tsh login --request-roles=hardware_key_requiredtsh sshrole.role_options.require_session_mfa: hardware_key_touch-tsh login --request-roles=hardware_key_touch_requiredtsh sshtsh logoutandtsh loginas the user with no hardware key requirement.auth_service.authentication.require_session_mfa: hardware_keytsh ls) should force automatic re-login with yubikeytsh sshauth_service.authentication.require_session_mfa: hardware_key_touchtsh ls) should force automatic re-login with yubikeytsh sshOther
Set
auth_service.authentication.require_session_mfa: hardware_key_touchin your cluster auth settings.tsh proxy db --tunnelHSM Support @tobiaszheller
Docs
Moderated session @tobiaszheller
Using
tshjoin an SSH session as two moderators (two separate terminals, role requires one moderator).Ctrl+Cin the Implement a prototype for a proxying SSH server that implements concepts expressed in readme #1 terminal should disconnect the moderator.Ctrl+Cin the Implement a functional prototype #2 terminal should disconnect the moderator and terminate the session as session has no moderator.Using
tshjoin an SSH session as two moderators (two separate terminals, role requires one moderator).tin any terminal should terminate the session for all participants.Performance @rosstimothy @fspmarshall @espadolini
Scaling Test
Scale up the number of nodes/clusters a few times for each configuration below.
Perform reverse tunnel node scaling tests for all backend configurations:
Perform the following additional scaling tests on DynamoDB:
Soak Test
Run 30 minute soak test directly against direct and tunnel nodes
and via label based matching. Tests should be run against a Cloud
tenant.
Concurrent Session Test
Run a concurrent session test that will spawn 5 interactive sessions per node in the cluster:
Robustness
resources which do not require a moderated session and in async recording
mode from an already issued certificate.
which require a moderated session and in async recording mode from an already
issued certificate.
are restarted.
Teleport with Cloud Providers
AWS @camscale
GCP @tigrato
IBM @hugoShaka
Application Access @mdwn
debug_app: trueworks.name.rootProxyPublicAddrand well aspublicAddr.name.rootProxyPublicAddr.app.session.startandapp.session.chunkevents are created in the Audit Log.app.session.chunkpoints to a 5 minute session archive with multipleapp.session.requestevents inside.tsh play <chunk-id>can fetch and print a session chunk archive.tsh apps login.tshcommands.tsh awstsh aws --endpoint-url(this is a hidden flag)tsh apps login.tsh azcommands.tsh proxy azandazcommands.tsh apps login.tsh gcloudcommands.tsh gsutilcommands.tsh proxy gcloudandgcloud/gsutilcommands.tctl create.tctl create -f.tctl rm.Add Applicationlinks to documentation.Database Access @smallinsky + team
select pg_sleep(10)followed by ctrl-c is a good query to test.)assume_role_arn: ""andexternal_id: "<id>"assume_role_arn: ""andexternal_id: "<id>"assume_role_arn: ""andexternal_id: "<id>"db.session.startis emitted when you connect.db.session.endis emitted when you disconnect.db.session.queryis emitted when you execute a SQL query.tsh db lsshows only databases matching role'sdb_labels.db_users.db_names.db.session.startis emitted when connection attempt is denied.db_names.db.session.queryis emitted when command fails due to permissions.tsh db connect.tctl create.tctl create -f.tctl rm.Please configure discovery in Discovery Service instead of Database Service.
assume_role_arnandexternal_idis set.- [ ] Test Databases screen in the web UI (tab is located on left side nav on dashboard): @GavinFrazar- [ ] Verify that all dbs registered are shown with correctname,description,type, andlabels- [ ] Verify that clicking on a rows connect button renders a dialogue on manual instructions withStep 2login value matching the rowsnamecolumn- [ ] Verify searching for all columns in the search bar works- [ ] Verify you can sort by all columns exceptlabelsTLS Routing @smallinsky
v2configuration starts only a single listener for proxy service, in contrast withv1configuration.Given configuration: @smallinsky
*:3080for proxy service. Given the configuration above, 3022 and 3025 will be opened for other services.v1, there should be additional ports 3023 and 3024.multiplexmodeauth_service.proxy_listener_mode: "multiplex"web_proxy_addr == tunnel_addrtsh db connectworks through proxy running inmultiplexmodetsh db proxywith a GUI client. @gabrielcorado @greedy52 @GavinFrazar @Tener @smallinsky (For each tested db protocol)multiplexmodessh -o "ForwardAgent yes" -o "ProxyCommand tsh proxy ssh" [email protected]ssh -o "ForwardAgent yes" -o "ProxyCommand tsh proxy ssh --user=%r --cluster=leaf-cluster %h:%p" [email protected]tsh sshaccess through proxy running in multiplex modemultiplexmodemultiplexmode behind L7 load balancer @Tenertsh loginandtctltsh sshandtsh configtsh proxy dbandtsh db connecttsh proxy appandtsh awstsh proxy kubeAssist
Assist is not supported by
tshand WebUI is the only way to use it.Assist test plan is in the core section instead of WebUI as most functionality is implemented in the core.
The text was updated successfully, but these errors were encountered: