0

I have an Ubuntu 20.04 Linux setup, with xfce, xorg and xrdp installed. My objective is to run an application inside a graphics environment, do some clicks, monitor its performance, and them close it after some time.

I am able to do it using an RDP session, by running "app.sh" in the terminal on the remote machine. But I would like this process to be more automated, and the app to start using remote ssh, whithout having to do the RDP session. Unfortunately I am not familiar enough with linux to know where to start.

What have I attempted so far:

  1. Just running "app.sh" from a remote ssh results in "Could not detect graphics environment" error.
  2. Using "xinit app.sh 'displayconfig'" command. The app seems to run but the app ui does not seem to initialize on the remote machine (or at least I cannot detect it)

For my purposes, I do not care whether or not a display session or rdp session is active (please correct me if I got the terminology wrong). My only objective is to start the app on the remote machine, send a click event to some (X,Y) coords, and then close the app. I do not want to actually see the actual UI.

Edit 1: Expected behavior (for clarity):

  1. I connect to the remote machine though ssh
  2. I run some command to start the app on the remote server
  3. The app starts on the remote server (with the ui and everithing)
  4. I dont see the UI on the local machine, but it is running on remote
  5. I send a click event to some (X,Y) coords (I have a python script for that)
  6. I monitor the performance of the app (the cloud setup does that for me)
  7. I close the app after some time
  8. I close the ssh connection

May be relevant: My app is a Java Swing based app, and I am using JVMTI to detect the UI

Edit 2: I will try to explain my setup a bit more, as the answers provided make the wrong assumptions:

I have 2 linux servers: server A and server B. server A acts as a repository (no graphics or anything) where different versions of the same app are uploaded. Once a new version for my app is uploaded, Server A uploaded that version to Server B, and deletes the old version (this part I have figured out). Now server B has some graphics installed. I need Server A to send some command to Server B to start the app as if it was a user (human) using the GUI. Now server A does not have any display, so I cannot forward the gui anywhere. So far, I have an 'app.sh' file with which managed to start the app, using RDP (from a windows machine) to connect. This, unfortunately, is not an option for production. What I need is some way to start a graphics environment on server B from server A and run the app there, or just have some 'always on' session which has a graphics environment on server B, and somehow start my app from server A with that session. I do not actually need the visual feed of the gui, as I dont need to see it, nor will I have any physical display for it, it can be purely virtual/simulated on server B, or it can be missing at all if there is a way to "click a button" without it.

Screenshots (redacted for confidentiality):

  1. App running with RDP session What you can see: Open RDP session, App running in background (behind terminal, not as background process), Terminal 2 started the app by calling 'app.sh', Terminal 1 sent a 'list ui' command to the app. All works fine

  2. SSH connection, attempting same commands as previously (remember, no PHISICAL display / no desktop screen available in production

What can you see: an open SSH connection, attempting to call same 'app.sh' as before.

More info: running 'echo $XDG_CURRENT_DESKTOP':

  1. in RDP session: > 'XFCE'
  2. in SSH session: > ''

Would something like this be possible? Please dont hesitate to request additional info

13
  • Is there a reason why you're using RDP? Couldn't you just ssh into the server while having an Xserver running on your local machine and then just start programs on the server, with them appearing locally on your machine? Have you tried this before going down the RDP path?
    – Ray
    Aug 22, 2022 at 13:37
  • @Ray I userd RDP just to test the setup, as I am familiar with it. As of local Xserver, yes, I have managed to do that, but this is the point, I dont want to see the ui locally. It can appear on the remote machine if it needs to, I just dont want to see it. I want a script which will start "in the background" in some sense, without opening any UI on my local machine and then write the result to a file. If the UI is actually running on the remote machine thats fine. I hope I am not to confusing :D Aug 22, 2022 at 13:45
  • Doesn't ssh -X yourhost ./app.sh (or wherever it is) work for you? It should display the application's gui on your local sccreen.
    – ubfan1
    Aug 22, 2022 at 16:05
  • No, it does not, as I do not actually want to see the gui locally. It may be there, on the remote machine, but I not want the visual feed. Aug 22, 2022 at 16:07
  • 1
    Sorry, but what you want is a bit confusing. If there is no GUI that appears, how do you interact with it? I mean, how will you "click" on it if you can't see it appear on your screen? What you're asking for sounds like it's a problem only the developer of that app can solve. Some programs have various arguments, and if you don't include them, it loads up the GUI. You basically need a program that has both a GUI and a command-line interface together in the same executable. No RDP or ssh with an XServer can solve that problem for you...
    – Ray
    Aug 22, 2022 at 16:22

2 Answers 2

1

Start ssh with: ssh -Y [email protected] and provide login credentials. Start X11 software with it's command, for example xclock (one might use & to preserve prompt). To start program in Windows 10/11 environment most recommendable is WSL2 with it's GUI support. Other option for Windows is Xming (https://sourceforge.net/projects/xming/). On Linux systems just ssh with -Y and start program. One has to enable X11 forwarding in target machine (where you start the program) in the /etc/ssh/sshd_config file by removing # in X11Forwarding yes (if no change to yes). Good practice is also enable forwarding in client (/etc/ssh/ssh_config) ForwardX11 yes. Remember to restart ssh by sudo systemctl restart ssh.

3
  • xclock returns "Error: Can't open display: ". xstart returns "(EE) no screens found(EE)", with "(EE) VESA(0): Cannot read int vect" in the logs Aug 22, 2022 at 18:00
  • startx seems to work with sudo, and starts the xorg process, but does not set any DISPLAY variable still. xclock still gives the same error with sudo. Aug 22, 2022 at 18:07
  • so: export DISPLAY=<the x11_display_you_have_in_that_machine_you_connected> and start.sh. Do you have X11 running? 1. startx 2. export DISPLAY 3. start script. Display info is in your /etc/X11 folder configuration files. Export DISPLAY= can take ip_of_the_connected_host or ip_of_server_you_connected:<display_number>. Display number in /etc/X11 conf files (don't remeber where there...). I would install VNC server and login (over SSH tunnel) with it into target, start the script and close vnc connection. That requires running X11 server though. Aug 22, 2022 at 22:18
0

From your comment,

I dont want to see the ui locally. It can appear on the remote machine if it needs to, I just dont want to see it. I want a script which will start "in the background" in some sense, without opening any UI on my local machine and then write the result to a file. If the UI is actually running on the remote machine thats fine.

It sounds like you want command-line control over software running on a remote machine, which has been used exclusively with GUI until this point, so that you can automate some task with that software. You want to execute commands on a remote machine, and receive the output.

It's impossible to know what the CLI commands are for that software without knowing what the software is, but the general structure would be this:

ssh user@remotemachine [command {flags, objects}] > /file/path/to/output/file/output.extension

For example, if I wanted to see the fstab contents on the remote computer at the IP address 123.45.67.890, then I would do the following:

ssh [email protected] cat /etc/fstab > /home/me/fstab.txt

For your case, you will need to find the command-line commands for the software you wish to use.

9
  • This is not quite my question. Sorry If It was not quite clear enough. I have an 'app.sh' file which is starting the app from console, and I am perfectly able to do that while being connected in an RDP session, which initiates the xfce graphics environment. The problem is that I dont want to connect to the RDP session, as I dont want to actually see the UI. But then I run 'app.sh' from a remote ssh, i get 'Could not detect graphics environment' error Aug 22, 2022 at 15:43
  • The reason you are receiving the "Could not detect graphics environment" error is because when you connect from your local machine to your remote machine via SSH, then tell your remote machine to run "app.sh", the remote machine tries to launch "app.sh" inside your user's session, which is active through SSH, and does not have a graphical environment configured/enabled. As a result you receive "Could not detect graphics environment", because there is no graphical environment to which app.sh can connect. Aug 22, 2022 at 15:51
  • What is app.sh doing, and what is your goal here? If you want the GUI to open on a remote machine, as if someone physically connected to the machine with a mouse, keyboard, and monitor, then why? Do you want it to be ready for the next person who logs into the physical machine? If so, why not have it launch at start, or at boot, or at login? Does the GUI provide text output to the console you need on your local machine? If so, then surely there is a CLI-only method to interface with the software, which you can execute remotely from your local machine. Aug 22, 2022 at 15:55
  • Yes. That seems reasonable. Hence the question, is there a way to provide this graphics environment? or maybe just have it "always running" and just point to it somehow? The RPD session seems to do this, as it runs xfce, but once I close the RDP, xfce also stops. And I dont actually want RDP because I dont need the visual feed, i just used it for testing. Or maybe just to have a different user session which is 'always on' which has a graphics environment, and I somehow start the app inside that session? Aug 22, 2022 at 15:59
  • To the question of why, I need to monitor how the app behaves when some GUI events happen. Simply put, I want to see "what happens when a button is clicked". Now I am able to detect the effect of the app working, but it has to come from the button being clicked, as if a user clicked it. Now I dont actually need to see the button being clicked, but it as to be an GUI click Aug 22, 2022 at 16:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .