2025-03-21 08:54:39 [scrapy.utils.log] INFO: Scrapy 2.9.0 started (bot: WalkoverCrawler) 2025-03-21 08:54:39 [scrapy.utils.log] INFO: Versions: lxml 4.9.3.0, libxml2 2.10.3, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.1, Twisted 22.10.0, Python 3.8.10 (default, Mar 17 2025, 19:35:05) - [GCC 9.4.0], pyOpenSSL 23.2.0 (OpenSSL 3.1.1 30 May 2023), cryptography 41.0.2, Platform Linux-5.15.0-1049-gcp-x86_64-with-glibc2.29 2025-03-21 08:54:39 [scrapy.crawler] INFO: Overridden settings: {'AUTOTHROTTLE_DEBUG': True, 'BOT_NAME': 'WalkoverCrawler', 'CONCURRENT_REQUESTS': 8, 'CONCURRENT_REQUESTS_PER_DOMAIN': 5, 'DOWNLOAD_DELAY': 5, 'FEED_EXPORT_ENCODING': 'utf-8', 'LOG_FILE': 'logs/WalkoverCrawler/Crawler/1ee715ec063211f0894b1b7b307a04c9.log', 'NEWSPIDER_MODULE': 'WalkoverCrawler.spiders', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'ROBOTSTXT_OBEY': True, 'SPIDER_MODULES': ['WalkoverCrawler.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'} 2025-03-21 08:54:39 [asyncio] DEBUG: Using selector: EpollSelector 2025-03-21 08:54:39 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor 2025-03-21 08:54:39 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop 2025-03-21 08:54:39 [scrapy.extensions.telnet] INFO: Telnet Password: 3ea4805836f05aa2 2025-03-21 08:54:39 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.logstats.LogStats'] 2025-03-21 08:54:39 [root] INFO: urls====>["https://httpstat.us/200?sleep=70000"] 2025-03-21 08:54:39 [root] INFO: {'job_id': '1ee715ec063211f0894b1b7b307a04c9', 'url': ['https://httpstat.us/200?sleep=70000'], 'company_id': '278513', 'bot_id': '686', 'status': 0} 2025-03-21 08:54:39 [root] INFO: endpoint:hello-chatbot-embedding-test-env.eba-ypmpzhyp.ap-south-1.elasticbeanstalk.com/api/update-status 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:39 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): r11.o.lencr.org:80 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:39 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): r11.o.lencr.org:80 2025-03-21 08:54:39 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): r11.o.lencr.org:80 2025-03-21 08:54:39 [urllib3.connectionpool] DEBUG: http://r11.o.lencr.org:80 "POST / HTTP/1.1" 200 504 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP response status: 2025-03-21 08:54:39 [urllib3.connectionpool] DEBUG: http://r11.o.lencr.org:80 "POST / HTTP/1.1" 200 504 2025-03-21 08:54:39 [urllib3.connectionpool] DEBUG: http://r11.o.lencr.org:80 "POST / HTTP/1.1" 200 504 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Verifying response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP response status: 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP response status: 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Responder is issuer 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Verifying response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Responder is issuer 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Verifying response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Responder is issuer 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Caching OCSP response. 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Caching OCSP response. 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Caching OCSP response. 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Using cached OCSP response. 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Using cached OCSP response. 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: Using cached OCSP response. 2025-03-21 08:54:39 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:39 [root] INFO: res: 2025-03-21 08:54:39 [root] INFO: Status: 200 2025-03-21 08:54:39 [WDM] INFO: ====== WebDriver manager ====== 2025-03-21 08:54:40 [WDM] INFO: Get LATEST chromedriver version for google-chrome 2025-03-21 08:54:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): googlechromelabs.github.io:443 2025-03-21 08:54:40 [urllib3.connectionpool] DEBUG: https://googlechromelabs.github.io:443 "GET /chrome-for-testing/latest-patch-versions-per-build.json HTTP/1.1" 200 10230 2025-03-21 08:54:40 [WDM] INFO: Get LATEST chromedriver version for google-chrome 2025-03-21 08:54:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): googlechromelabs.github.io:443 2025-03-21 08:54:40 [urllib3.connectionpool] DEBUG: https://googlechromelabs.github.io:443 "GET /chrome-for-testing/latest-patch-versions-per-build.json HTTP/1.1" 200 10230 2025-03-21 08:54:40 [WDM] INFO: Get LATEST chromedriver version for google-chrome 2025-03-21 08:54:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): googlechromelabs.github.io:443 2025-03-21 08:54:40 [urllib3.connectionpool] DEBUG: https://googlechromelabs.github.io:443 "GET /chrome-for-testing/latest-patch-versions-per-build.json HTTP/1.1" 200 10230 2025-03-21 08:54:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): googlechromelabs.github.io:443 2025-03-21 08:54:41 [urllib3.connectionpool] DEBUG: https://googlechromelabs.github.io:443 "GET /chrome-for-testing/known-good-versions-with-downloads.json HTTP/1.1" 200 83685 2025-03-21 08:54:41 [WDM] INFO: WebDriver version 118.0.5993.70 selected 2025-03-21 08:54:41 [WDM] INFO: Modern chrome version https://storage.googleapis.com/chrome-for-testing-public/118.0.5993.70/linux64/chromedriver-linux64.zip 2025-03-21 08:54:41 [WDM] INFO: About to download new driver from https://storage.googleapis.com/chrome-for-testing-public/118.0.5993.70/linux64/chromedriver-linux64.zip 2025-03-21 08:54:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): storage.googleapis.com:443 2025-03-21 08:54:41 [urllib3.connectionpool] DEBUG: https://storage.googleapis.com:443 "GET /chrome-for-testing-public/118.0.5993.70/linux64/chromedriver-linux64.zip HTTP/1.1" 200 8269742 2025-03-21 08:54:41 [WDM] INFO: Driver downloading response is 200 2025-03-21 08:54:43 [WDM] INFO: Get LATEST chromedriver version for google-chrome 2025-03-21 08:54:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): googlechromelabs.github.io:443 2025-03-21 08:54:43 [urllib3.connectionpool] DEBUG: https://googlechromelabs.github.io:443 "GET /chrome-for-testing/latest-patch-versions-per-build.json HTTP/1.1" 200 10230 2025-03-21 08:54:44 [WDM] INFO: Driver has been saved in cache [/home/khangori850/.wdm/drivers/chromedriver/linux64/118.0.5993.70] 2025-03-21 08:54:44 [selenium.webdriver.common.service] DEBUG: Started executable: `/home/khangori850/.wdm/drivers/chromedriver/linux64/118.0.5993.70/chromedriver-linux64/chromedriver` in a child process with pid: 3129321 2025-03-21 08:54:44 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://localhost:40047/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--headless", "--no-sandbox", "--disable-dev-shm-usage", "--remote-debugging-port=9222"]}}}} 2025-03-21 08:54:44 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): localhost:40047 2025-03-21 08:54:46 [urllib3.connectionpool] DEBUG: http://localhost:40047 "POST /session HTTP/1.1" 200 0 2025-03-21 08:54:46 [selenium.webdriver.remote.remote_connection] DEBUG: Remote response: status=200 | data={"value":{"capabilities":{"acceptInsecureCerts":false,"browserName":"chrome","browserVersion":"118.0.5993.117","chrome":{"chromedriverVersion":"118.0.5993.70 (e52f33f30b91b4ddfad649acddc39ab570473b86-refs/branch-heads/5993@{#1216})","userDataDir":"/tmp/.org.chromium.Chromium.7GIV3K"},"fedcm:accounts":true,"goog:chromeOptions":{"debuggerAddress":"localhost:9222"},"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platformName":"linux","proxy":{},"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify","webauthn:extension:credBlob":true,"webauthn:extension:largeBlob":true,"webauthn:extension:minPinLength":true,"webauthn:extension:prf":true,"webauthn:virtualAuthenticators":true},"sessionId":"6360739607f7a7ff87f5acb490b2b942"}} | headers=HTTPHeaderDict({'Content-Length': '848', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}) 2025-03-21 08:54:46 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request 2025-03-21 08:54:46 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy_selenium_custom.SeleniumMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2025-03-21 08:54:46 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-03-21 08:54:46 [scrapy.middleware] INFO: Enabled item pipelines: ['WalkoverCrawler.pipelines.WalkovercrawlerPipeline'] 2025-03-21 08:54:46 [scrapy.core.engine] INFO: Spider opened 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): r11.o.lencr.org:80 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): r11.o.lencr.org:80 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): r11.o.lencr.org:80 2025-03-21 08:54:46 [urllib3.connectionpool] DEBUG: http://r11.o.lencr.org:80 "POST / HTTP/1.1" 200 504 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP response status: 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Verifying response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Responder is issuer 2025-03-21 08:54:46 [urllib3.connectionpool] DEBUG: http://r11.o.lencr.org:80 "POST / HTTP/1.1" 200 504 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP response status: 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Verifying response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Responder is issuer 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Caching OCSP response. 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:46 [urllib3.connectionpool] DEBUG: http://r11.o.lencr.org:80 "POST / HTTP/1.1" 200 504 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP response status: 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Verifying response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Responder is issuer 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Caching OCSP response. 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Caching OCSP response. 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:46 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2025-03-21 08:54:46 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Using cached OCSP response. 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Using cached OCSP response. 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: Using cached OCSP response. 2025-03-21 08:54:46 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:54:47 [scrapy.core.engine] DEBUG: Crawled (404) (referer: None) 2025-03-21 08:54:47 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://localhost:40047/session/6360739607f7a7ff87f5acb490b2b942/url {"url": "https://httpstat.us/200?sleep=70000"} 2025-03-21 08:55:58 [urllib3.connectionpool] DEBUG: http://localhost:40047 "POST /session/6360739607f7a7ff87f5acb490b2b942/url HTTP/1.1" 200 0 2025-03-21 08:55:58 [selenium.webdriver.remote.remote_connection] DEBUG: Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}) 2025-03-21 08:55:58 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request 2025-03-21 08:56:08 [selenium.webdriver.remote.remote_connection] DEBUG: GET http://localhost:40047/session/6360739607f7a7ff87f5acb490b2b942/source {} 2025-03-21 08:56:08 [urllib3.connectionpool] DEBUG: http://localhost:40047 "GET /session/6360739607f7a7ff87f5acb490b2b942/source HTTP/1.1" 200 0 2025-03-21 08:56:08 [selenium.webdriver.remote.remote_connection] DEBUG: Remote response: status=200 | data={"value":"\u003Chtml>\u003Chead>\u003Cmeta name=\"color-scheme\" content=\"light dark\">\u003C/head>\u003Cbody>\u003Cpre style=\"word-wrap: break-word; white-space: pre-wrap;\">200 OK\u003C/pre>\u003C/body>\u003C/html>"} | headers=HTTPHeaderDict({'Content-Length': '220', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}) 2025-03-21 08:56:08 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request 2025-03-21 08:56:08 [selenium.webdriver.remote.remote_connection] DEBUG: GET http://localhost:40047/session/6360739607f7a7ff87f5acb490b2b942/url {} 2025-03-21 08:56:08 [urllib3.connectionpool] DEBUG: http://localhost:40047 "GET /session/6360739607f7a7ff87f5acb490b2b942/url HTTP/1.1" 200 0 2025-03-21 08:56:08 [selenium.webdriver.remote.remote_connection] DEBUG: Remote response: status=200 | data={"value":"https://httpstat.us/200?sleep=70000"} | headers=HTTPHeaderDict({'Content-Length': '47', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}) 2025-03-21 08:56:08 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request 2025-03-21 08:56:08 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2025-03-21 08:56:08 [scrapy.extensions.logstats] INFO: Crawled 2 pages (at 2 pages/min), scraped 0 items (at 0 items/min) 2025-03-21 08:56:08 [pymongo.ocsp_support] DEBUG: Peer did not staple an OCSP response 2025-03-21 08:56:08 [pymongo.ocsp_support] DEBUG: Requesting OCSP data 2025-03-21 08:56:08 [pymongo.ocsp_support] DEBUG: Trying http://r11.o.lencr.org 2025-03-21 08:56:08 [pymongo.ocsp_support] DEBUG: Using cached OCSP response. 2025-03-21 08:56:08 [pymongo.ocsp_support] DEBUG: OCSP cert status: 2025-03-21 08:56:08 [root] INFO: Item not dropped bze status code is 200: 2025-03-21 08:56:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://httpstat.us/200?sleep=70000> None 2025-03-21 08:56:08 [root] INFO: {'job_id': '1ee715ec063211f0894b1b7b307a04c9', 'url': ['https://httpstat.us/200?sleep=70000'], 'company_id': '278513', 'bot_id': '686', 'status': 1} 2025-03-21 08:56:08 [root] INFO: endpoint:hello-chatbot-embedding-test-env.eba-ypmpzhyp.ap-south-1.elasticbeanstalk.com/api/update-status 2025-03-21 08:56:09 [root] INFO: res: 2025-03-21 08:56:09 [root] INFO: Status: 200 2025-03-21 08:56:09 [scrapy.core.engine] INFO: Closing spider (finished) 2025-03-21 08:56:09 [selenium.webdriver.remote.remote_connection] DEBUG: DELETE http://localhost:40047/session/6360739607f7a7ff87f5acb490b2b942 {} 2025-03-21 08:56:09 [urllib3.connectionpool] DEBUG: http://localhost:40047 "DELETE /session/6360739607f7a7ff87f5acb490b2b942 HTTP/1.1" 200 0 2025-03-21 08:56:09 [selenium.webdriver.remote.remote_connection] DEBUG: Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}) 2025-03-21 08:56:09 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request 2025-03-21 08:56:09 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 221, 'downloader/request_count': 1, 'downloader/request_method_count/GET': 1, 'downloader/response_bytes': 671, 'downloader/response_count': 2, 'downloader/response_status_count/200': 1, 'downloader/response_status_count/404': 1, 'elapsed_time_seconds': 82.864446, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2025, 3, 21, 8, 56, 9, 245193), 'item_scraped_count': 1, 'log_count/DEBUG': 135, 'log_count/INFO': 31, 'memusage/max': 92688384, 'memusage/startup': 92688384, 'response_received_count': 2, 'robotstxt/request_count': 1, 'robotstxt/response_count': 1, 'robotstxt/response_status_count/404': 1, 'scheduler/dequeued': 1, 'scheduler/dequeued/memory': 1, 'scheduler/enqueued': 1, 'scheduler/enqueued/memory': 1, 'start_time': datetime.datetime(2025, 3, 21, 8, 54, 46, 380747)} 2025-03-21 08:56:09 [scrapy.core.engine] INFO: Spider closed (finished)