chg: [valkey] get rid of Redis and moved to Valkey
Some checks are pending
Black formatting / Black (push) Waiting to run
ShellCheck / ShellCheck (push) Waiting to run

This commit is contained in:
Alexandre Dulaunoy 2024-11-23 15:57:18 +01:00
parent 9cc2bef2bc
commit facc75d06d
Signed by: adulau
GPG key ID: 09E2CD4944E6CBCD
5 changed files with 17 additions and 17 deletions

View file

@ -5,12 +5,12 @@ be used against [cve-search](https://github.com/cve-search/cve-search) to do act
## Requirements ## Requirements
- Redis - [Valkey](https://valkey.io/)
- Python - Python
## Usage ## Usage
To use CPE guesser, you have to initialise the Redis database with `import.py`. To use CPE guesser, you have to initialise the [Valkey](https://valkey.io/) database with `import.py`.
Then you can use the software with `lookup.py` to find the most probable CPE matching the keywords provided. Then you can use the software with `lookup.py` to find the most probable CPE matching the keywords provided.
@ -28,7 +28,7 @@ If you don't want to install it locally, there is a public online version. Check
### Docker ### Docker
#### Single image with existing Redis #### Single image with existing Valkey
```bash ```bash
docker build . -t cpe-guesser:l.0 docker build . -t cpe-guesser:l.0
@ -157,9 +157,9 @@ sharing common names or name is composed of multiple words.
Split vendor name and product name (such as `_`) into single word(s) and then canonize the word. Building an inverse index using Split vendor name and product name (such as `_`) into single word(s) and then canonize the word. Building an inverse index using
the cpe vendor:product format as value and the canonized word as key. Then cpe guesser creates a ranked set with the most common the cpe vendor:product format as value and the canonized word as key. Then cpe guesser creates a ranked set with the most common
cpe (vendor:product) per version to give a probability of the CPE appearance. cpe (vendor:product) per version to give a probability of the CPE appearance.
### Redis structure ### Valkey structure
- `w:<word>` set - `w:<word>` set
- `s:<word>` sorted set with a score depending of the number of appearance - `s:<word>` sorted set with a score depending of the number of appearance

View file

@ -1,3 +1,3 @@
redis valkey[libvalkey]
falcon falcon
dynaconf dynaconf

View file

@ -8,7 +8,7 @@ import urllib.request
import gzip import gzip
import shutil import shutil
import xml.sax import xml.sax
import redis import valkey
import time import time
from dynaconf import Dynaconf from dynaconf import Dynaconf
@ -16,7 +16,7 @@ from dynaconf import Dynaconf
settings = Dynaconf(settings_files=["../config/settings.yaml"]) settings = Dynaconf(settings_files=["../config/settings.yaml"])
cpe_path = settings.cpe.path cpe_path = settings.cpe.path
cpe_source = settings.cpe.source cpe_source = settings.cpe.source
rdb = redis.Redis(host=settings.redis.host, port=settings.redis.port, db=8) rdb = valkey.Valkey(host=settings.valkey.host, port=settings.valkey.port, db=8)
class CPEHandler(xml.sax.ContentHandler): class CPEHandler(xml.sax.ContentHandler):

View file

@ -1,6 +1,6 @@
server: server:
port: 8000 port: 8000
redis: valkey:
host: 127.0.0.1 host: 127.0.0.1
port: 6379 port: 6379
cpe: cpe:

View file

@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
import redis import valkey
from dynaconf import Dynaconf from dynaconf import Dynaconf
# Configuration # Configuration
@ -10,9 +10,9 @@ settings = Dynaconf(settings_files=["../config/settings.yaml"])
class CPEGuesser: class CPEGuesser:
def __init__(self): def __init__(self):
self.rdb = redis.Redis( self.rdb = valkey.Valkey(
host=settings.redis.host, host=settings.valkey.host,
port=settings.redis.port, port=settings.valkey.port,
db=8, db=8,
decode_responses=True, decode_responses=True,
) )
@ -35,4 +35,4 @@ class CPEGuesser:
rank = self.rdb.zrank("rank:cpe", cpe) rank = self.rdb.zrank("rank:cpe", cpe)
ranked.append((rank, cpe)) ranked.append((rank, cpe))
return sorted(ranked) return sorted(ranked, reverse=True)