Appsmith behind upstream proxy

Hello.

I installed appsmith on ubuntu that has an upstream proxy.
It doesn’t work and I get this error:
reactor.core.Exceptions$ErrorCallbackNotImplemented: io.netty.channel.ConnectTimeoutException: connection timed out: api64.ipify.org/108.171.202.195:443
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: at <had to remove bc forum rule for new user>
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe$2.run(AbstractEpollChannel.java:575)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
|_ checkpoint ⇢ Request to GET https://api64.ipify.org [DefaultWebClient]

I can use curl fine since I have the http_proxy and https_proxy env vars set up.

Is there a way to configure appsmith to use the proxy too?

Thanks.

Hi

Are you running Appsmith in an air-gapped environment or with limited internet connectivity? If that’s the case, then the Community Edition of Appsmith won’t run in this environment.

Air-gapped environments will be supported in our Enterprise editions only. We plan to release an early preview of the Enterprise edition by end of Dec 2021 or early 2022.

Hi @arpit_appsmith. I have full internet connectivity. I just have a proxy. As I said curl works fine.

I actually got further on and changed supervisor configuration to start java with -Dhttp.proxy settings.

But I got another similar error while accessing cs.appsmith.com:

 - connection timed out: cs.appsmith.com/18.119.45.4:443
 - An exception was thrown by reactor.netty.resources.PooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer.operationComplete()
reactor.core.Exceptions$ErrorCallbackNotImplemented: io.netty.channel.ConnectTimeoutException: connection timed out: cs.appsmith.com/18.119.45.4:443
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: cs.appsmith.com/18.119.45.4:443
        at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe$2.run(AbstractEpollChannel.java:575)
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
        |_ checkpoint ⇢ Request to GET null [DefaultWebClient]
Stack trace:
                at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe$2.run(AbstractEpollChannel.java:575)
                at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
                at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170)
                at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
                at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
                at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
                at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
                at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
                at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
                at java.base/java.lang.Thread.run(Thread.java:829)

I don’t understand the reason for this particular error… it seems to me that a proxy is used in most organizations…

Notice how this error seems to not know the request url:
“checkpoint ⇢ Request to GET null [DefaultWebClient]”

just to ping @arpit_appsmith

Hi

I think your proxy is blocking calls to *.appsmith.com. Currently, the community version doesn’t support environments where these network calls are blocked. The reason we make these networks calls are:

  1. Fetching the changelog for our releases.
  2. Checking if there is a new release available for auto-updating.
  3. Being able to provide support if you tag @appsmith-support in any comment on any UI widget.

If you whitelist *.appsmith.com, Appsmith should work for you. If you are concerned about data leakages, you can read our documentation on telemetry to see the data that is sent in any of these network calls. TL;DR: We don’t send any private data to our servers.

Let me know if this works for you.

@arpit_appsmith thanks for looking into this.

But that is not our case - we have full internet connectivity, no blocking rules.
As you can see I can reach cs.appsmith.com fine from the docker host.
The problem here is something else.

wget http://cs.appsmith.com/
--2021-10-20 08:49:07--  http://cs.appsmith.com/
Connecting to 192.168.2.1:8080... connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: https://cs.appsmith.com:443/ [following]
--2021-10-20 08:49:08--  https://cs.appsmith.com/
Connecting to 192.168.2.1:8080... connected.
Proxy request sent, awaiting response... 401 Unauthorized

As I reported, I cannot find a documented way to configure appsmith to use a proxy to access external urls. I solved this by changing the java startup script.

Still, I get this error while accessing cs.appsmith.com in appsmith logs.

As I said, the host can access this url just fine by using wget or curl.

So my question is really:

  • how to configure appsmith to use an “normal” upstream proxy?

I’m hoping that by configuring appsmith correctly the error will be fixed too.

Thanks,
Duarte

Hey Duarte

Apologies that this is taking a little back & forth to resolve.

Is the Docker daemon configured to use the HTTP & HTTPS proxy variables? I’m wondering if services running inside the Docker container are bypassing the proxy while the host machine is adhering to the proxy settings.

As a confirmation, have you taken a look at this article: Configure Docker to use a proxy server | Docker Documentation ? Is the Docker daemon configured to route all HTTP calls via the proxy server? Maybe if we configure the proxy settings on the Docker daemon itself, we don’t need to modify the startup script in Java.

Best

Arpit

Hi @arpit_appsmith. Yes I have configured docker to be aware of the proxy.

I have just now solved the issue… I forgot to set env var no_proxy, so I think some requests were timing out for that reason.
Now appsmith starts and shows the gui in the browser!

To sum up:
1 - configure docker-compose.yml to set proxy env vars, adding a new section environment:
environment:
http_proxy: http://192.168.2.1:8080/
https_proxy: http://192.168.2.1:8080/
no_proxy: 127.0.0.1, localhost, 172.17.0.1,172.18.0.1,10.230.70.115, 192.68.2.64,192.168.2.84,192.168.2.20,internaldomain.pt

2 - configure java command to use the proxy, by changing /etc/supervisor/conf.d/beckend.conf:
command=java -Dhttp.proxyHost=192.168.2.1 -Dhttp.proxyPort=8080 -Dhttp.nonProxyHosts=“localhost,127.0.0.1,172.17.0.1,172.18.0.1,172.18.0.2,172.18.0.3,10.230.70.115, 192.68.2.64,192.168.2.84,192.168.2.20,edia.pt” -Dhttps.proxyHost=192.168.2.1 -Dhttps.proxyPort=8080 -Dhttps.nonProxyHosts=“localhost,127.0.0.1,172.17.0.1,172.18.0.1,172.18.0.2,172.18.0.3,10.230.70.115, 192.68.2.64,192.168.2.84,192.168.2.20,internaldomain.pt” -Dserver.port=8080 -Djava.security.egd=‘file:/dev/./urandom’ -jar server.jar

Now things start fine.

The problem now is that every time appsmith container is updated, java configuration is lost and has to be redone…

Thanks for taking the time.

Hi have news, after a call with support.

The problem seems related to somehow appsmith docker container not reaching cs(dot)appsmith(dot)com.
Looking at the logs, when starting up, after logging in, the browser requests /api/v1/applications/new.
Appsmith proceeds to get something from cs(dot)appsmith(dot)com which timesout. Only after this timeout does appsmith respond to the browser.
The browser has given up before the reply, and shows an error message “server took too long to respond” and just gives up. On the screen you can only see the skeleton where data would appear…

To solve: just adding a line to /etc/hosts pointing cs(dot)appsmith(dot)com to a local/reachable ip solves the problem.

To do this automatically, you can add a new section to appsmith docker-compose.yml:

services:
  appsmith:
    image: index.docker.io/appsmith/appsmith-ce
    container_name: appsmith
    (...)
    extra_hosts:
    - "cs(dot)appsmith(dot)com:local_ip"

You can also do this by creating a docker-compose-override.yml, but I haven’t tested this (Share Compose configurations between files and projects | Docker Documentation).