Personal tools
In other languages
|
CMSFAQFrom techmeetIndymedia Content Management System (CMS)More related Informations about the CMS... ... and the Background of the Network and the needs. Frequently Asked Question (FAQ)This FAQ tries to be a summary for the discussion of the last couple years, to help new developers to understand the requirements and goals of the new cms. Why does Indymedia need a new CMS?
What is the current status of the CMS?After two years of discussion, we are still at the scratch-board and had a long research of all kind of possibilities. Make sure you have a look around in the CMSSurveyList. That said, we have come a long way and we are currently in the planning stages of creating a prototype / proof-of-concept. San Francisco IMC has indicated that they would deploy this proof-of-concept publicly in order to see what happens! Why it took so long to develop this?Because before first techmeet in 2006 we received the notices that the Ahimsa servers will stop giving services for Indymedia websites and that the California Community Colocation Project (CCCP) will shutdown and we have to find new places and hardware to host more than 25 Indymedia global sites and a lot of other groups and social movements. Then at the beggining of 2007 the research for the new-cms started. What is the outcome of the Research?At the end of the reasearch we had three groups:
DrupalThere is and was a active group of Drupal hackers, even before the whole CMS Discussion in 2006. There are a couple of IMCs who are using Drupal. For some small IMCs, Drupal is a very good choice, since it is easy to customize and does not need any big programming experience for customizing. If you know Drupal, join the kids and help them. IMC Drupal Development PloneIt would be possible to use Plone to recreate the current features of a Mir based Indymedia Site. It would even use the same amount of Server Resources. More informations on the Plone Wiki Site. PuzzleThis approach is the idea to use different existing Tools, Frameworks and CMS'es to create a complete Site Setup that fits out requirements. We started to write a proof-of-concept code to show this approach. However, it is hard to explain this approach in a few words. In combination with the proof-of-concept we went to Techmeet 2008 in Rio to explain the idea. There is a video/audio recoding of the Presentation available. 2008-09-07: Update from Rio: Presentation is done and the recording is made: We will provide downloads for the recording in the next days. What are the Requirements for the new CMS?There is a CMSWhatWeWant Feature Request List from 2006 that will give you a good overview of the needed Front-end functions. However, there are more requirements and ideas for the back-end and the general structure of the CMS. We will try to address these ideas and requirements in this FAQ. Why a new CMS?If you read the Request List, you might get the impression that there are at least 10 CMS out there that would make it possible to create a Indymedia Site. However, there are 4 requirements that no CMS out there can fulfill.
We are aware of the problems that come with a new CMS. Without a large group of developers, we will end up in the same situation as indymedia is now. The old codebases had a very specific functionality, special purpose, and the lack of good documentation made it hard to use them for anything but indymedia. With MIR we have a flexible software, but it was never used outside of Indymedia because of the different approach and hard setup. We are being very careful to only choose technology that has an existing, large, non-IMC developers community behind it. Why should I separate the CMS over many servers?First of all, it happens that the State and Police is taking away our servers. Using many servers in many country's makes it harder to take down the whole network at once. Having many different servers with different resources creates a big need for a very flexible CMS to optimize the usage of the servers. In addition, this model prevents the network being harmed by not only server seizures but also hardware failures and donation withdrawals (i.e. ahimsa). Why not use the normal Cache and Back-end Setup?We already use squid, varnishd and mod_proxy together with the old PHP based CMS, but this setup is not fail-safe at all. As soon as you take down the back-end server, the whole website is down. A cache is good for fixing bandwidth or speed issues, but it is a bad choice for creating a autonomous system that will continue to deliver informations as long as possible. A cache always works on a request based system. The user asks for content X and the cache is requesting the content only at that very moment from the back-end. This way it is very unlikely that the cache will ever have a full copy of all possible content objects. Since the cache can not determine all possible request, it can not make a synchronization with the back-end. However, the back-end is able to create and update a "cache", or better "a local copy", that contains all possible content objects. With the "local copy" the system can deliver the content without depending on the back-end. The only problem is to find a transport from the back-end to the front-end server that will keep the "local copy" up-to-date. Finding a good transport solution for this problem is part of the imc-cms project goals. How would you split a CMS over different servers?You can separate the functionality of a CMS into different categories.
So the idea is, that each of these modules will continue to work, even if the database and back-end server is down, they should have there own autonomy. How does that look like in detail for......static Output?Output of static content like images should be no problem. Each of these static output servers will have a complete copy of all static files. It would also be possible to create servers based on the type of the content (audio, video, images etc). This gives more flexibility for the usage of the bandwidth and disk space. It is also a big help to have the text content as static .shtml Websites available. This makes it possible to continue to share all informations even if the database and back-end is offline. We had moments in the network where we just edited .shtml files on a couple of mirrors, to keep the users informed about the outage of the back-end server. Try that with any kind of cache. For more informations about this topic, read the Producer Wiki: [1] ...dynamic Output?Well, you will run into some problems here, that will have a impact on the structure and functionality of the web-site.
... User InteractionWell, here you would implement all the possible write operations via CGI and AJAX Interfaces. How can two server exchange static filesSmall FilesIndymedia is using rsync together with the [CMSProducer Mir Producter] for many many years now. However, it does not allow a nearly real-time (up to 10 seconds) update of the content.
TODO 2008-07-28: Both WebDAV and CouchDB need further research and testing. Large Filesv2v.cc has a python based implementation of a Video Distribution Network. It allows fast distribution of large files over many Servers using a torrent based network for the transfer. How can two server exchange content informations ?Well, the most common way to exchange informations between servers are "Remote Procedure Calls" RCP like SOAP or XML-RPC. However, they are very limited in the functionality and the XML Overhead don't make them lightweight. There are some Enterprise Solutions called Middleware and they Provide Functions we need and you would have to re implement with RPC.
What Middleware would you use?We stumbled over the Middleware Solution ICE from ZeroC and hope it will be a good choice. There is still a need to have a closer research of ICE, to make sure a possible setup would scale and be fast. How does ICE work?ICE makes functions call over the network layer - so called "remote procedure calls" short RPC. What is the difference between ICE and SOAP / XML-RPC?With SOAP/XML-RPC you can do result = remote_server.function(parameter1,parameter2) With ICE you can do object.value = 1 result = remote_server.function(object) This allows a much better integration into a existing OO Model. There are many more differences and features of ICE, but please, check the ICE Homepage for that. Example ICE CodeHere is some basic pseudo python code that shows how ICE works. Host A calls function publish on Host B with a article as parameter. On Host B it should print out "Hello World". Note: This will not work as copy/paste code, it is really just pseudo code to show how ICE can be used. Code Part 1ICE Interface Definition. From this so called slice file you auto-generate the Article and ArticleDist class with the help of a ICE command line tool. Code Part 2Host A ICE code (acting as ICE client)
a = Article()
a.title = "Hello World"
ic=Ice.initialize()
base = ic.stringToProxy("ArticleDist:default -p 10000")
dist=CDN.ArticleDistPrx.checkedCast(base)
dist.publish(article)
Full Code publisher/article.py Code Part 3Host B ICE code (acting as ICE server providing the "publish" function) The Class->Function that will be called.
class ArticleDistI(CDN.ArticleDist):
def publish(self, article, current=None):
print article
Loading and starting ICE
ic =Ice.initialize(sys.argv)
adapter=ic.createObjectAdapterWithEndpoints(
"ArticleDistAdapter", "default -p 10000")
object=ArticleDistI()
adapter.add(object,Ice.stringToIdentity("ArticleDist"))
adapter.activate()
ic.waitForShutdown()
How would you integrate a Middleware into a CMS?You can insert the Middleware over different Layers of your application. ICE as DB ModelIf you think of the Model View and Controller, the Middleware would be a Layer in Front of the Model. There is more general information here at dev.bunke.indymedia.org and at DesignInspiration. ICE as PublisherLets have a look at this Graph http://cyberpunk.cl/malandro/03.jpg "publish" is the view, frontend the controller. What we add now, is a abstraction layer called "publisher". The Workflow for creating new content looks like this:
Proof of ConceptPlease watch / listen to the presentation at Techmeet 2008 - it will explain many many of your questions. The Server StructureThere are couple of graphs who show the general idea of "splitting" a CMS into different parts. Front-endhttp://pineiro.linefeed.org/ http://dev.bunke.indymedia.org/wiki/TechOverview Back-endhttps://codecoop.org/projects/malandro/ More InformationsWhere can I find some code?There are two projects by now that came out of the CMS Discussion. Cake StuffHyperActiveA Ruby on Rails IMC Site that shows new Features like Events and Places. It was build for IMC Denmark, but gives you a good Overview of the possible Look of a new IMC CMS. Malandro
Malandro is the first code that is using the ICE Middleware. At the moment (April 2008) it is not more then a raw skeleton of classes to show the structure and setup of the new CMS. However, it is possible to add new content, so you can see how all components are working together. Malandro is using:
How do i get in contact?
Who is participating ?Feel free and add yourself to the List! :) Views |
