Mining Guide

Miners operating multiple machines with the same API key should avoid sending multiple requests simultaneously, as this may lead to rate-limiting.

Welcome to the Market Compass Subnet 17 Mining Guide! In this guide, you'll learn how to set up and deploy a mining node on Subnet 17 to contribute to the data gathering process for Market Compass. Let's get started:

Prerequisites

Before diving into the mining process, ensure you have the necessary environment variables set up:

MC_BEARER_TOKEN: Bearer token for Twitter Pro API. Export this variable with your X (Twitter) API token.
```
export MC_BEARER_TOKEN=AAA...
```
MC_SUBNET_API_X_API_KEY: Market Compass API key for validators. Contact us via Discord-ticket on the Subnet 17 Discord Server to obtain a new API key.
```
export MC_SUBNET_API_X_API_KEY=<your-api-key>
```
MC_SUBNET_API_URL: URL to the current version of the used API. Set this variable to the appropriate API URL.
```
export MC_SUBNET_API_URL=https://api3.subnet.marketcompass.ai
```

Dependencies

Ensure you have the necessary dependencies installed. The key dependency is the CommuneX library. You can find the full dependency list in the requirements.txt file. Here are the main dependencies:

communex
typer
uvicorn
keylimiter
pydantic-settings

Miner Setup

Follow these steps to set up and register your miner on Subnet 17:

Run Miner: From the root of your project, run the following command to start the miner:

comx module serve subnet.miner.model.Miner <name-of-your-com-key> --subnets-whitelist 17 --ip 0.0.0.0 --port 8000

Register Miner: Register your miner using the following command:

comx module register <name-of-your-miner> <name-of-your-com-key> --ip <your-ip-of-the-server> --port 8000 --netuid 17

Miner Code

import os
import requests

from communex.module import Module, endpoint
from communex.key import generate_keypair
from keylimiter import TokenBucketLimiter


class Miner(Module):
    def __init__(self):
        super().__init__()
        self.bearer_token = os.getenv('MC_BEARER_TOKEN')

    @endpoint
    def generate(self, prompt: str, start_time: str = '2024-04-01T5:00:00Z', max_results: int = 50):
        # TODO: pass start_time, max_results from validator
        url = "https://api.twitter.com/2/tweets/search/all"

        def bearer_oauth(r):
            r.headers["Authorization"] = f"Bearer {self.bearer_token}"
            r.headers["User-Agent"] = "v2FullArchiveSearchPython"
            return r

        response = requests.request("GET", url, auth=bearer_oauth, params={
            'query': prompt,
            "max_results": max_results,
            "start_time": start_time,
            "user.fields": "id,username,name",
            "tweet.fields": "created_at,author_id"
        })

        if response.ok:
            tweets = response.json()
            return tweets['data']
        raise Exception(f"Cant get tweets for prompt '{prompt}'")


if __name__ == "__main__":
    from communex.module.server import ModuleServer
    import uvicorn

    key = generate_keypair()
    miner = Miner()
    refill_rate = 1 / 400
    bucket = TokenBucketLimiter(2, refill_rate)
    server = ModuleServer(miner, key, ip_limiter=bucket, subnets_whitelist=[17])
    app = server.get_fastapi_app()

    # Only allow local connections
    uvicorn.run(app, host="0.0.0.0", port=8000)

Validator Setup

To validate and register your miner, follow these steps:

Run Validator: Execute the validator script to initiate validation:
```
python3 -m market-compass-subnet.subnet.cli <name-of-your-com-key>
```

Register Validator: Register your validator using the following command:

comx module register market-compass::<your-vali-name> <name-of-your-com-key> --netuid 17

import os
import requests

from communex.module import Module, endpoint
from communex.key import generate_keypair
from keylimiter import TokenBucketLimiter


class Miner(Module):
    def __init__(self):
        super().__init__()
        self.bearer_token = os.getenv('MC_BEARER_TOKEN')

    @endpoint
    def generate(self, prompt: str, start_time: str = '2024-04-01T5:00:00Z', max_results: int = 50):
        # TODO: pass start_time, max_results from validator
        url = "https://api.twitter.com/2/tweets/search/all"

        def bearer_oauth(r):
            r.headers["Authorization"] = f"Bearer {self.bearer_token}"
            r.headers["User-Agent"] = "v2FullArchiveSearchPython"
            return r

        response = requests.request("GET", url, auth=bearer_oauth, params={
            'query': prompt,
            "max_results": max_results,
            "start_time": start_time,
            "user.fields": "id,username,name",
            "tweet.fields": "created_at,author_id"
        })

        if response.ok:
            tweets = response.json()
            return tweets['data']
        raise Exception(f"Cant get tweets for prompt '{prompt}'")


if __name__ == "__main__":
    from communex.module.server import ModuleServer
    import uvicorn

    key = generate_keypair()
    miner = Miner()
    refill_rate = 1 / 400
    bucket = TokenBucketLimiter(2, refill_rate)
    server = ModuleServer(miner, key, ip_limiter=bucket, subnets_whitelist=[17])
    app = server.get_fastapi_app()

    # Only allow local connections
    uvicorn.run(app, host="0.0.0.0", port=8000)

Validator Setup

To validate and register your miner, follow these steps:

Run Validator: Execute the validator script to initiate validation:

python3 -m market-compass-subnet.subnet.cli <name-of-your-com-key>

Register Validator: Register your validator using the following command:

comx module register market-compass::<your-vali-name> <name-of-your-com-key> --netuid 17

Further Reading

For more information and documentation on the Commune AI ecosystem and subnet details, please visit the Official Commune Page and its developer documentation.