You are here

IPFS for file storage?

2 posts / 0 new
Last post
IPFS for file storage?
Hey Guys,
Im currently working on a web/game project of my own - and I came across a technology called 'IPFS' (Interplanetary File System)
It is built on the 'Bittorrent' technology - but it is designed with the intent of being a Decentralized and censor free internet.
Basically - the way it works is that when you run an IPFS node - you are simultaneously donating X amount of file storage space to the network,
while at the same time - getting access to the total network in order to store and distribute your own content.
When you add a file to the IPFS - that file can then be requested via any IPFS node around the world.
If the node does not have the file, it sends a message out to 50+ node/peers it is connected to for the file and it repeats this request for those 50+ node/peers until it finds the file in question - usually the first node to have the file, is your own node - but it then shares it with any node that requests it.
As the file travels across the world via each peer, a copy of the file is left behind - further increasing the availability of the file - meaning your node can eventually go offline and the file will remain accessible.
So - I am aware that The Neverwinter Vault is a site that is either funded by the community or possibly out of the Pocket of Rolo Kipp (perhaps my intel is years old...)
Providing file storage space for GB's and GB's of content can be somewhat expensive - depending on the provider.
I was wondering if IPFS would be something the Vault would benefit from?
I have integrated IPFS into my website forum - when someone uploads an image via the Forum, the image goes into a temp database.
While the image content is available in my DB - I will serve the image to the viewer on the website from my db.
However- within 15 seconds of the file being added to the DB, another task is executed to take the image content from the db and add it to the IPFS network.
Once the IPFS network has acknowledged the images data, and the image is retrievable from the IPFS network, I then delete my database copy.
Attempts to resolve the image are then done so directly from the IPFS network - without me having to host the image files myself.
Voila - infinite file storage space.
Some caveats do exist -
Files on the IPFS apparently automatically get 'de-prioritized' if they are not accessed for a long time. Similar to the way torrents work - if less people have the file, its harder to get the file. However, the theory is that as long as at least one person has the file, then the file is always available.
Im not sure there is a real way to 'delete' content on the IPFS - once it is on the network, it is kinda permanent and static.
You can upload newer versions of the file sure - but each one will generate a new HASH which generates a new download location url etc.
Some benefits also exist
If you add a file to the IPFS and it already exists - it will not be added twice.
The hash that you use to locate files on the network, is derived from the content of the file.
So you will in effect avoid duplicate files on the network - makes it lean etc.
The network also works for large files too - no real size limit.
I've read articles where people have uploaded 3.5 gb or more in single file uploads to the network.

In similar way people provide the MD5s and sha256 hashes to verify files - the Filename/hash used to request the file on the network acts as a form of automatic signing - Meaning the content cannot be tampered with and will always be what you expect it to be.
Just to demonstrate - if you were to run an IPFS node, and request QmasNEXBtxQ66wDTMZmWDx4Y7RPcBLnA18p1V368ToibQc
You would be able to download an Empty CEP2_Custom.hak file that I just added to the network.
Or - if you don't want to run a node to download/get access to the files - you can use the website, which acts like a public portal to the network.   (just save it with a .hak extension to make it usable)
Note - if you navigate to it, it will compain about encoding - just paste into browser window and press return to download it. Then change extension to .hak
Although - I don't know what restrictions they have on bandwidth usage etc
So its prob not a good idea to rely on their website for a production system.
Although - I think its acceptable to use their website as a means of propagating your file across the network.
Remember - the file will not actually travel anywhere, unless someone requests it.
Requesting the file via their website, will pull the file from your system, onto their node, and then make it available in 2 locations, instead of just on your node.
The use case I am using this system for in my website/game is for the storage of Music, Videos, Images and Lua scripts that users will upload.
I will use my bandwidth to serve these files back to the users, but I do not want to use my file-storage space to host the files.
So my website in effect acts like a portal to the IPFS network just like the site above.
The image on this forum topic for instance (and all the forum topics for that matter) - are actually being served from the IPFS network.
Each time someone views the image - actually preserves its life on the network.
BTW- my website is still in development - I am basically building a Forum system from scratch.
And then the website itself will be a portal to an online story based game engine.
  • up
  • down
Also - Wikipedia is an example of a website that has been hypothesized about being added to the IPFS - as a way of getting Wikipedia into countries that prevent access. So - something like the Lexicon - being a static website an all - would be a good candidate to add in its entirety to the IPFS network.
  • up
  • down