DInference | A decentralised LLM inferencing infrastructure

Project Name: DInference | A decentralised LLM inferencing infrastructure

Project Track: web3 (DePin)

Team Name: wizards

Team Member(s): anand

Devpost Project Link: https://devpost.com/software/dinference-a-decentralised-llm-inferencing-infrastructure

Project Goal: The goal of dInference is to establish a decentralized infrastructure for Large Language Model (LLM) inferencing, facilitating broader access to computational resources for individuals and organizations. Through incentivizing GPU providers with dInference tokens, the project aims to democratize LLM inferencing capabilities, fostering innovation, preventing centralization of power, and creating a resilient and collaborative ecosystem in the web3 space.

Project Value: The primary goal of dInference is to create a decentralized infrastructure for large language model (LLM) inferencing, enabling individuals and organizations to contribute their GPU resources and earn rewards in the form of dInference tokens. By democratizing access to LLM inferencing capabilities, dInference aims to prevent the concentration of power in the hands of a few major companies and foster an open and collaborative ecosystem.

Project Info: dInference offers significant value by addressing the following challenges:

  1. Accessibility: LLM inferencing requires substantial computational resources, which can be prohibitively expensive for many individuals and organizations. dInference enables broader access to these capabilities by leveraging the idol GPU resources of its community.

  2. Decentralization: By distributing the inferencing workload across a decentralized network of GPU providers, dInference mitigates the risk of centralized control and promotes a more resilient and transparent ecosystem.

  3. Incentivization: The dInference token incentivizes GPU providers to contribute their resources, ensuring a sustainable and self-regulating ecosystem. Additionally, the token burn mechanism helps maintain the token’s scarcity and value.

  4. Innovation: By democratizing access to LLM inferencing, dInference fosters innovation and encourages the development of new applications and use cases, potentially leading to breakthroughs in various domains.

![Settings page to buy credits or get API Keys|690x373]
(upload://u96NT0OsamSmX2bT0A83kb8fxeg.jpeg)

Project Website: https://dinference.vercel.app/

Project Test Instructions: to be updated

Project Details: dInference is built on top of the following core components:

  1. dInference Platform: A decentralized web application that serves as the hub for GPU providers and end-users. GPU providers can register their URLs and monitor their earnings, while end-users can access and utilize the available LLM inference APIs.

  2. dInference Docker Image: A self-created Docker image that encapsulates the necessary dependencies and configurations to run various LLM models. GPU providers can download and run this image on their GPU machines in a few steps and get the LLM inferencing APIs.

  3. Load Balancing and API Gateway: A load balancing and API gateway component that distributes incoming end-user requestsPreformatted text to the registered GPU providers based on availability and performance metrics.

  4. Token Economy: The dInference token serves as the incentive mechanism for GPU providers. A portion of the fees paid by end-users is used to buy and burn dInference tokens, increasing their scarcity and value.

Smart Contract links TRONSCAN | TRON BlockChain Explorer | 波场区块链浏览器

Project Milestones:

  1. Phase 1: Launch of the dInference platform, including the web application and easy deployment solution for gpu providers.
  2. Phase 2: Introducing Token mechanism for incentivizing GPU providers
  3. Phase 3: Optimization of the load balancing and API gateway components for improved performance and scalability.
  4. Phase 4: Expansion of the dInference ecosystem through partnerships and collaborations, fostering adoption and innovation.
6 Likes

Welcome to Grand hackathon season 6
Please what kind of GPU resources are users to be donated? I think there is Gaming GPU , cloud base GPU and so on , please I need more enlightment on that area

2 Likes

Your vision of democratizing access to large language model (LLM) inferencing through decentralized infrastructure is both innovative and impactful. To understand your project better, can you elaborate on the mechanism through which individuals and organizations can contribute GPU resources to the dInference network?

2 Likes

Welcome to season 6, I will follow up once more details are added.Good luck buddy

3 Likes

You are welcome but I am lost, know nothing about this lol.
All the best

2 Likes

Welcome to Grand hackathon season 6

  • How will dInference ensure the security of user data and prevent malicious actors from injecting code or manipulating results on the platform.

  • Can the dInference network handle a large number of concurrent requests and different LLM model sizes.

  • How will dInference ensure compatibility with various LLM models and frameworks.

  • How will dInference maintain the quality of inference results generated by potentially unreliable or deverse GPU provide.

1 Like

Hi @Nweke-nature1.com
This is not about donating the idol GPUs to others, rather a platform that enables users to monetize their idol GPUs by running LLMs provided by us.
We have curated some open-source LLMs on our side which we allow users to run on their systems in a few steps and they can register the URLs for their local LLMs on our platform. They will be paid tokens for running these LLMs.

1 Like

Hello @Chizz thanks for your kind words. We are enabling users to monetize their idol GPUs by running LLMs provided by us. They will be paid tokens for running these LLMs.
While the end-users can come and user the LLM APIs for their own/official use without relying on any centralised entity and for cheaper prices. We will act as an mediator between the LLM APIs provider and users

Sure @Relate101 we will be son releasing our website

Hello @Prince-Onscolo I can surely explain in more detail if you have any queries. Rest assured, you will understand things very well once we release a detailed video of our platform

ok what is this project all about?

2 Likes

Hello @Okorie you have asked some really valid and appropriate question, will try to answer each concern in order.

  • The providers are provided with docker images from our platform having the LLM and their inferencing code loaded. The providers have to run those images to get the valid endpoints, other endpoints will not be valid

  • Yes, we certainly can. We do not assign a URL registered by provider directly to end user rather pass through our load balancer which keeps on checking the active servers and forward requests to those.

  • As said everything is there in the docker image itself, providers do not have to change anything on their end

  • We will actively checking the server status of all the providers every few minutes to ensure they are active as well as to ensure they are running the valid servers only.

Let me know if I could solve your concerns well.

Hello, we want to build a platform that acts as a mediator between idol GPU holders and end users who want LLM APIs. we ask people having idol GPUs to run LLMs provided by us and in return they keep getting our token.
The end users can use these APIs for their official works at low costs. This was both the parties are happy.

1 Like

Thanks for your response, we will wait

do you mean idle or idol? meaning of that “Idol”

Oh I see, please how can someone who has no knowledge about this get involved? Is there any range or specific amount for rewards, in simple terms how does the rewards looks like

2 Likes

Your project is vert complex for a non techy like me to understand, can you please explain in simple 2 lines?

Welcome to Hackathon Season 6, your offering is really interesting. I see the project DInference aims to establish a decentralized infrastructure for large language model (LLM) inferencing, offering accessibility, decentralization, incentivization, and innovation.

How does the tokenomics model balance incentivization with token scarcity and value stability?

1 Like

Welcome to the Hackathon, I have read everything, please tell me how are you going to carry out security and privacy of the user data, thank you

1 Like

Thank you for this info. How will contributors be rewarded for contributing their GPUs resources?