Skip to content

Failing to start  #153

Open
Open

Description

@mrwhoo

Hello,

CentOS 7
Docker: Docker version 19.03.5, build 633a0ea
ES, Kibana - 7.5.1

Any tips ... google is not helpful!

Thank you in advance.

@bitsensor/elastalert@2.0.1 start /opt/elastalert-server
sh ./scripts/start.sh

00:26:05.086Z INFO elastalert-server: Config: No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
00:26:05.088Z INFO elastalert-server: Config: Proceeding to look for normal config file.
00:26:05.089Z INFO elastalert-server: Config: A config file was found in /opt/elastalert-server/config/config.json. Using that config.
00:26:05.109Z INFO elastalert-server: Router: Listening for GET request on /.
00:26:05.110Z INFO elastalert-server: Router: Listening for GET request on /status.
00:26:05.110Z INFO elastalert-server: Router: Listening for GET request on /status/control/:action.
00:26:05.110Z INFO elastalert-server: Router: Listening for GET request on /status/errors.
00:26:05.110Z INFO elastalert-server: Router: Listening for GET request on /rules.
00:26:05.111Z INFO elastalert-server: Router: Listening for GET request on /rules/:id.
00:26:05.112Z INFO elastalert-server: Router: Listening for POST request on /rules/:id.
00:26:05.112Z INFO elastalert-server: Router: Listening for DELETE request on /rules/:id.
00:26:05.112Z INFO elastalert-server: Router: Listening for GET request on /templates.
00:26:05.112Z INFO elastalert-server: Router: Listening for GET request on /templates/:id.
00:26:05.112Z INFO elastalert-server: Router: Listening for POST request on /templates/:id.
00:26:05.112Z INFO elastalert-server: Router: Listening for DELETE request on /templates/:id.
00:26:05.112Z INFO elastalert-server: Router: Listening for POST request on /test.
00:26:05.112Z INFO elastalert-server: Router: Listening for GET request on /config.
00:26:05.112Z INFO elastalert-server: Router: Listening for POST request on /config.
00:26:05.113Z INFO elastalert-server: Router: Listening for POST request on /download.
00:26:05.113Z INFO elastalert-server: Router: Listening for GET request on /metadata/:type.
00:26:05.113Z INFO elastalert-server: Router: Listening for GET request on /mapping/:index.
00:26:05.113Z INFO elastalert-server: Router: Listening for POST request on /search/:index.
00:26:05.116Z INFO elastalert-server: ProcessController: Starting ElastAlert
00:26:05.116Z INFO elastalert-server: ProcessController: Creating index
00:26:05.513Z INFO elastalert-server:
ProcessController: Elastic Version:7
Mapping used for string:{'type': 'keyword'}
Index elastalert_status already exists. Skipping index creation.

00:26:05.513Z INFO elastalert-server: ProcessController: Index create exited with code 0
00:26:05.513Z INFO elastalert-server: ProcessController: Starting elastalert with arguments [none]
00:26:05.520Z INFO elastalert-server: ProcessController: Started Elastalert (PID: 36)
00:26:05.521Z INFO elastalert-server: Server: Server listening on port 3030
00:26:05.522Z INFO elastalert-server: Server: Websocket listening on port 3333
00:26:05.522Z INFO elastalert-server: Server: Server started
00:26:06.142Z ERROR elastalert-server:
ProcessController: Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main

00:26:06.142Z ERROR elastalert-server:
ProcessController: "main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/opt/elastalert/elastalert/elastalert.py", line 1929, in
sys.exit(main(sys.argv[1:]))
File "/opt/elastalert/elastalert/elastalert.py", line 1925, in main

00:26:06.142Z ERROR elastalert-server:
ProcessController: client.start()
File "/opt/elastalert/elastalert/elastalert.py", line 1106, in start

00:26:06.143Z ERROR elastalert-server:
ProcessController: self.run_all_rules()
File "/opt/elastalert/elastalert/elastalert.py", line 1158, in run_all_rules

00:26:06.143Z ERROR elastalert-server:
ProcessController: self.send_pending_alerts()
File "/opt/elastalert/elastalert/elastalert.py", line 1534, in send_pending_alerts

00:26:06.143Z ERROR elastalert-server:
ProcessController: pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit)
File "/opt/elastalert/elastalert/elastalert.py", line 1526, in find_recent_pending_alerts

00:26:06.144Z ERROR elastalert-server:
ProcessController: size=1000)
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped

00:26:06.144Z ERROR elastalert-server:
ProcessController: return func(*args, params=params, **kwargs)
TypeError: search() got an unexpected keyword argument 'doc_type'

00:26:06.174Z ERROR elastalert-server: ProcessController: ElastAlert exited with code 1
00:26:06.174Z INFO elastalert-server: Server: Stopping server
00:26:06.175Z INFO elastalert-server: ProcessController: ElastAlert is not running
00:26:06.175Z INFO elastalert-server: Server: Server stopped. Bye!`

Activity

  1. spacecabbie commented on Jan 28, 2020

    @spacecabbie

    Same here has to do with new version that was launched this week seeing last update in the repo might be a while.

  2. alexshurik commented on Feb 11, 2020

    @alexshurik

    Same. Any updates?

  3. MisderGAO commented on Feb 14, 2020

    @MisderGAO

    same problem. container can not be started

  4. deric commented on Feb 18, 2020

    @deric

    See Yelp/elastalert, Python 3 is required for Yelp/elastalert > 0.2
    If you want to use Python 2, you should build this image with e.g.:

    make build v=v0.1.39
    

    The version was bumped here. Yes, the naming and versioning of these two projects is confusing...

  5. rbkumar88 commented on Mar 2, 2020

    @rbkumar88

    getting the following error if i try to build using the above command,

    Installed /usr/lib/python2.7/site-packages/configparser-0.0.0-py2.7.egg
    error: The 'configparser>=3.5.0' distribution was not found and is required by elastalert
    

    Also tried to run with the already hosted image "bitsensor/elastalert:2.0.0" which uses v0.1.39 build of elastalert I believe and getting the following error

    npm info it worked if it ends with ok
    npm verb cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'start', '--verbose' ]
    npm info using npm@6.9.0
    npm info using node@v12.2.0
    npm verb run-script [ 'prestart', 'start', 'poststart' ]
    npm info lifecycle @bitsensor/elastalert@2.0.1~prestart: @bitsensor/elastalert@2.0.1
    npm info lifecycle @bitsensor/elastalert@2.0.1~start: @bitsensor/elastalert@2.0.1
    
    > @bitsensor/elastalert@2.0.1 start /opt/elastalert-server
    > sh ./scripts/start.sh
    
    19:12:31.345Z  INFO elastalert-server: Config:  No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
    19:12:31.346Z  INFO elastalert-server: Config:  Proceeding to look for normal config file.
    19:12:31.347Z  INFO elastalert-server: Config:  A config file was found in /opt/elastalert-server/config/config.json. Using that config.
    19:12:31.360Z  INFO elastalert-server: Router:  Listening for GET request on /.
    19:12:31.360Z  INFO elastalert-server: Router:  Listening for GET request on /status.
    19:12:31.361Z  INFO elastalert-server: Router:  Listening for GET request on /status/control/:action.
    19:12:31.361Z  INFO elastalert-server: Router:  Listening for GET request on /status/errors.
    19:12:31.361Z  INFO elastalert-server: Router:  Listening for GET request on /rules.
    19:12:31.362Z  INFO elastalert-server: Router:  Listening for GET request on /rules/:id.
    19:12:31.362Z  INFO elastalert-server: Router:  Listening for POST request on /rules/:id.
    19:12:31.362Z  INFO elastalert-server: Router:  Listening for DELETE request on /rules/:id.
    19:12:31.362Z  INFO elastalert-server: Router:  Listening for GET request on /templates.
    19:12:31.362Z  INFO elastalert-server: Router:  Listening for GET request on /templates/:id.
    19:12:31.362Z  INFO elastalert-server: Router:  Listening for POST request on /templates/:id.
    19:12:31.363Z  INFO elastalert-server: Router:  Listening for DELETE request on /templates/:id.
    19:12:31.366Z  INFO elastalert-server: Router:  Listening for POST request on /test.
    19:12:31.366Z  INFO elastalert-server: Router:  Listening for GET request on /config.
    19:12:31.366Z  INFO elastalert-server: Router:  Listening for POST request on /config.
    19:12:31.366Z  INFO elastalert-server: Router:  Listening for POST request on /download.
    19:12:31.366Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/:type.
    19:12:31.366Z  INFO elastalert-server: Router:  Listening for GET request on /mapping/:index.
    19:12:31.366Z  INFO elastalert-server: Router:  Listening for POST request on /search/:index.
    19:12:31.368Z  INFO elastalert-server: ProcessController:  Starting ElastAlert
    19:12:31.368Z  INFO elastalert-server: ProcessController:  Creating index
    19:12:32.150Z  INFO elastalert-server:
        ProcessController:  Elastic Version:7
        Mapping used for string:{'type': 'keyword'}
        Index elastalert already exists. Skipping index creation.
        
    19:12:32.150Z  INFO elastalert-server: ProcessController:  Index create exited with code 0
    19:12:32.151Z  INFO elastalert-server: ProcessController:  Starting elastalert with arguments [none]
    19:12:32.160Z  INFO elastalert-server: ProcessController:  Started Elastalert (PID: 51)
    19:12:32.161Z  INFO elastalert-server: Server:  Server listening on port 3030
    19:12:32.162Z  INFO elastalert-server: Server:  Websocket listening on port 3333
    19:12:32.162Z  INFO elastalert-server: Server:  Server started
    19:12:32.163Z ERROR elastalert-server:
        Server:  Error creating data folder with error: [Error: EROFS: read-only file system, mkdir '/opt/elastalert/server_data'] {
          errno: -30,
          code: 'EROFS',
          syscall: 'mkdir',
          path: '/opt/elastalert/server_data'
        }
    (node:28) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'testFolder' of undefined
        at /opt/elastalert-server/src/controllers/test/index.js:18:64
    (node:28) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
    (node:28) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
    19:12:32.813Z ERROR elastalert-server:
        ProcessController:  Traceback (most recent call last):
          File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
            "__main__", fname, loader, pkg_name)
          File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
            exec code in run_globals
          File "/usr/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1929, in <module>
        
    19:12:32.814Z ERROR elastalert-server:
        ProcessController:      sys.exit(main(sys.argv[1:]))
          File "/usr/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1925, in main
            client.start()
          File "/usr/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1106, in start
            self.run_all_rules()
          File "/usr/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1158, in run_all_rules
        
    19:12:32.815Z ERROR elastalert-server:
        ProcessController:      self.send_pending_alerts()
          File "/usr/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1534, in send_pending_alerts
        
    19:12:32.815Z ERROR elastalert-server:
        ProcessController:      pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit)
          File "/usr/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1526, in find_recent_pending_alerts
        
    19:12:32.816Z ERROR elastalert-server:
        ProcessController:      size=1000)
          File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
            return func(*args, params=params, **kwargs)
        TypeError: search() got an unexpected keyword argument 'doc_type'
        
    19:12:32.857Z ERROR elastalert-server: ProcessController:  ElastAlert exited with code 1
    19:12:32.857Z  INFO elastalert-server: Server:  Stopping server
    19:12:32.857Z  INFO elastalert-server: ProcessController:  ElastAlert is not running
    19:12:32.857Z  INFO elastalert-server: Server:  Server stopped. Bye!
    npm verb lifecycle @bitsensor/elastalert@2.0.1~start: unsafe-perm in lifecycle true
    npm verb lifecycle @bitsensor/elastalert@2.0.1~start: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/opt/elastalert-server/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    npm verb lifecycle @bitsensor/elastalert@2.0.1~start: CWD: /opt/elastalert-server
    npm info lifecycle @bitsensor/elastalert@2.0.1~poststart: @bitsensor/elastalert@2.0.1
    npm verb exit [ 0, true ]
    npm timing npm Completed in 3023ms
    npm info ok 
    
  6. smallersoup commented on Mar 26, 2021

    @smallersoup

    same problem. container can not be started

    Following is my pod.yaml

    kind: Pod
    apiVersion: v1
    metadata:
      name: es-alert-server-0
      generateName: es-alert-server-
      namespace: cdc
      labels:
        app: es-alert-server
        controller-revision-hash: es-alert-server-6d89cdf5d4
        statefulset.kubernetes.io/pod-name: es-alert-server-0
      annotations:
        kubesphere.io/containerSecrets: '{"es-alert-server":"demo-registry-secret"}'
        kubesphere.io/restartedAt: '2021-03-26T10:33:46.549Z'
        logging.kubesphere.io/logsidecar-config: '{}'
    spec:
      volumes:
        - name: host-time
          hostPath:
            path: /etc/localtime
            type: ''
        - name: volume-3y9242
          persistentVolumeClaim:
            claimName: es-alert-data
        - name: volume-y3r2wv
          configMap:
            name: elastalert-test-yaml
            defaultMode: 420
        - name: volume-3c4pip
          configMap:
            name: elastalert-yaml
            defaultMode: 420
        - name: volume-qvzigq
          configMap:
            name: es-alert-config
            defaultMode: 420
        - name: volume-013ds1
          secret:
            secretName: es-cdc-a-cert
            defaultMode: 420
        - name: default-token-hg8h5
          secret:
            secretName: default-token-hg8h5
            defaultMode: 420
      containers:
        - name: es-alert-server
          image: 'harbor-xadd.staff.xdf.cn/library/karql-elastalert:4.0.0'
          command:
            - /bin/sh
            - '-c'
            - >-
              sed -i 's|10000|60000|' /opt/elastalert-server/src/common/websocket.js
              && npm start
          ports:
            - name: tcp-3030
              containerPort: 3030
              protocol: TCP
            - name: tcp-3333
              containerPort: 3333
              protocol: TCP
          resources:
            limits:
              cpu: '1'
              memory: 1000Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
            - name: host-time
              readOnly: true
              mountPath: /etc/localtime
            - name: volume-3y9242
              mountPath: /opt/elastalert
            - name: volume-y3r2wv
              readOnly: true
              mountPath: /opt/elastalert/config-test.yaml
              subPath: config-test.yaml
            - name: volume-3c4pip
              readOnly: true
              mountPath: /opt/elastalert/config.yaml
              subPath: config.yaml
            - name: volume-qvzigq
              readOnly: true
              mountPath: /opt/elastalert-server/config/config.json
              subPath: config.json
            - name: volume-013ds1
              readOnly: true
              mountPath: /certs
            - name: default-token-hg8h5
              readOnly: true
              mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      serviceAccountName: default
      serviceAccount: default
      nodeName: master03
      securityContext: {}
      imagePullSecrets:
        - name: demo-registry-secret
      hostname: es-alert-server-0
      subdomain: es-alert-server-z93a
      affinity: {}
      schedulerName: default-scheduler
      tolerations:
        - key: node.kubernetes.io/not-ready
          operator: Exists
          effect: NoExecute
          tolerationSeconds: 300
        - key: node.kubernetes.io/unreachable
          operator: Exists
          effect: NoExecute
          tolerationSeconds: 300
      priority: 0
      enableServiceLinks: true
    
    
    
    
    08:44:48.601Z INFO elastalert-server: Server: Server started
    
    08:44:48.603Z ERROR elastalert-server:
    
    Server: Error creating data folder with error: [Error: EACCES: permission denied, mkdir '/opt/elastalert/server_data'] {
    
    errno: -13,
    
    code: 'EACCES',
    
    syscall: 'mkdir',
    
    path: '/opt/elastalert/server_data'
    
    }
    

    rewrite docker image command to : sleep 10000, then exec container ,see following :

    /opt/elastalert $ pwd
    /opt/elastalert
    /opt/elastalert $ mkdir aa
    mkdir: can't create directory 'aa': Permission denied
    /opt/elastalert $ ls -l
    total 16
    drwxrwxrwx    2 root     root             6 Mar 26 09:20 config-test.yaml
    drwxrwxrwx    2 root     root             6 Mar 26 09:20 config.yaml
    drwx------    2 root     root         16384 Mar 21 16:27 lost+found
    /opt/elastalert $ whoami
    node
    /opt/elastalert $ cd ..
    /opt $ ls -l
    total 4
    drwxr-xr-x    5 root     root          4096 Mar 21 16:27 elastalert
    drwxr-xr-x    1 node     node            16 Mar 26 09:20 elastalert-server
    drwxr-xr-x    1 node     node            80 Mar 12 12:00 yarn-v1.22.5
    /opt $
    

    user: node creates a directory in root's directory: elastalert

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions