EUIYOUNG HWANG

Euiyoung Hwang’s Profile

Euiyoung Hwang (marieuig@gmail.com) / Linkedin(https://www.linkedin.com/in/euiyoung-hwang/) : Search, Data & AI Software Engineer

I have ten years of experience in working with a modern search platform (Elasticsearch, Google Search Appliance with Google Apps) and in building data pipelines(e.g https://github.com/euiyounghwang/python-search) & rest api services around it as a search engineer/senior software engineer. Especially, I am an expert in the Search Engine with a bunch of api from elasticsearch and rest_api environment using Python Web Stacks with Docker because I have been handling the entire version of ES from 1.7 up to 7.9 (i.e building ES cluster from the scrach, handling index mappings with a lof of analyzers and resolving complex query based on domain needs)

In FiscalNote (2022.07 ~ 2023.07), I contributed to improve search relevance with complex queries such as func_score to adjust the weight to search results and query performance with clusters. In more detail, I did the following:

In particular, I remember that I have been building and implementing ‘Enterprise Search Services’ for seven years based on Elasticsearch in South Korea. As a result of my contributions that I was having a success story interview at the elastic on seoul conference(https://www.youtube.com/watch?v=qu0IXwi3Fq0). At that time, I participated in the Google search engine replacement project(https://www.linkedin.com/pulse/elastic-tour-seoul-posco-ict-euiyoung-hwang/) as project leader and senior software engineer.

The screenshots attached below are for the Success Story with Elasticsearch Interview, Elastic on Seoul Conference, 2018 (https://www.elastic.co/customers/posco) when i worked as Senior Software Engineer & Search/Data Engineer at POSC ICT, South Korea (Received an award in POST ICT, 2016, https://media.licdn.com/dms/image/C512DAQGqaGMRMAXk9w/profile-treasury-image-shrink_1920_1920/0/1597560813194?e=1694487600&v=beta&t=sYbj3Kip8j_opHS_GB2ECOQ0FVhoiv16Jgsb2dxHp1M)

Alt text

if you want to watch the video, please go to this url (https://www.youtube.com/watch?v=qu0IXwi3Fq0) after set subtitled to English (I was in the middle of guys)

Alt text

Recently, I am personally implementing to Rest-Api Endpoint as test projects using python, flask/fastapi(https://github.com/euiyounghwang/python-fastapi-vector-search, https://github.com/euiyounghwang/python-flask-connexion-example-openapi3-master), and nestjs(https://github.com/euiyounghwang/nest-js-rest-api). The service allows you to search from a search engine (elasticsearch) and Postgres. It is also implemented based on Docker, and is being built, executed, and tested. Also I am interested with similary search such as huggingface embedding, vectorized search using Faiss and so on. (https://github.com/euiyounghwang/semantic-search-elasticsearch-openai-langchain)

Rest-API on OPEN API Specifiation(Swagger)

components:
  schemas:
    ..
    Search:
      type: object
      properties:
        query_string:
          type: string
          description: Full text search
          default: "Cryptocurrency"
          nullable: true
        start_date:
          type: string
          format: date
          description: Start date
          default: "2021 01-01 00:00:00"
        size:
          type: integer
          description: The number of size
          default: 20
        sort_order:
          type: string
          enum:
            - DESC
            - ASC
        include_basic_aggs:
          type: boolean
          description: Flag to enable/disabled aggregations which can slow down queries
        pit_id:
          type: string
          format: date
          description: pit_id
          example: ""
        ids_filter:
           type: array
           items:
            type: string
           default: ["*"]
    ..

Alt text

Docker in my local Environment

Alt text

I have set up & tested to monitor with alert throught slack such as search engines, restapi endpoints and other application’s metrics using prometheus, alertmanager and grafana. Elasticsearch Prometheus Exporter is a builtin exporter from Elasticsearch to Prometheus. It collects all relevant metrics and makes them available to Prometheus via the Elasticsearch REST API. This is an open source project - Cluster status, Node Status such as JVM, Indices, Circuit Breaker : the feature to prevent OOM occurrence Alt text

loading at once and evaluate the rule periodically based on ‘evaluation_interval’

rule_files:

  # My local environment to install node-exporter Docker instance
  # docker run --rm -p 9100:9100 prom/node-exporter 
  # docker compose up -d node-exporter
  node_exporter:
    # http://localhost:9100/metrics
    image: prom/node-exporter
    container_name: node_exporter
    depends_on:
      - prometheus
    restart: always
    ports:
    - 9100:9100

Alt text

Prometheus.yml

Prometheus (Build Docker on my local environment with Elasticsearch-Exporter/Python-Export/FastAPI Plugin to gather all relevant metrics)

Alt text

Python Webservice monitoring

Elastic Stack Monitoring

Alt text

/usr/local/bin/metricbeat -e –path.home=/home/devuser/ES/metricbeat-8.8.0-linux-arm64

– sudo vi /etc/systemd/system/metricbeat.service

[Unit] Description=Metricbeat Service After=multi-user.target

[Service] Type=simple User=devuser Group=devuser WorkingDirectory=/home/devuser/ES/metricbeat-8.8.0-linux-arm64 #ExecStart=/home/devuser/ES/metricbeat-8.8.0-linux-arm64b/start_metricbeat.sh ExecStart=/usr/local/bin/metricbeat -e –path.home=/home/devuser/ES/metricbeat-8.8.0-linux-arm64 Restart=on-failure

[Install] WantedBy=multi-user.target

– Service Registry sudo systemctl daemon-reload

Autostart when rebooting

sudo systemctl enable metricbeat.service

start to service

sudo systemctl start metricbeat.service sudo systemctl status metricbeat.service

– Log check journalctl -u metricbeat.service ``` Alt text