This emerging technology that is called Grid Computing allows (as I mentioned before) anyone to call his computer(s) grid. No one is restricted, in naming (thanks god), but it is confusing. "What do you mean when you say GRID?"
A lot of Grid projects and software exists. Some of it SOA based and some not. This industry gives jobs to a lot of people, so it is probably a good thing. If grid is the up-to-date version of the MF computer, then Greg Matter was right - THE WORLD NEEDS ONLY FIVE COMPUTERS but somehow, I feel there are other ways to see things. If the famous Alice and Bob have an apple each they one can say "they have 2 apples", but one can imagine 3 groups that each one has apples:
(1) Alice
(2) Bob
(3) "Alice and Bob".
This example can be presented "intro to combinatorics" under grad course.
Since each computer can be used for different tasks at the same time, it can be certainly be used for different projects. And if one can use the same computers for several tasks, the computer will be counted in several different projects. So the same as when we counted apples. There are as many as the number of sub-gorups and the same goes for computers.
Long time ago computers had CPU(s), memory and peripherals. A modern computer is composed of network and interaction device (keyboard, mouse, screen). Thats it !
The processor, memory, storage are not part of the modern computer. Neither are the applications, and the peripherals. They or their functionality can be found out there. Most of the resources are free but cost a lot (I will explain it later). Network, applications, compute power and storage can be found for free by resource providers. "blogger.com" is an example of application provider and so does second life (that offers interactive TV for the first time in history). One can broadcast to (almost) all the world (NBC, BBC, CNN - be aware!). And the same when my computer is connected to youtube, I am part of the international cross boundary cinema.
"But waht about business - they keep their computers restricted. You can never know what happens inside..." - Wrong again. When your application, calls Amazon to order books and software, they do some work for you.
As a matter of fact, Grid and SOA is all about outsource your IT !
"Does it mean fire my IT people in the long run ?" - Not at all. They will work on a different environment and provide different services. Any internet shop runs on part of Amazon, Google, Yahoo, etc. The business process (at last) is the main issue of the IT. No more self sustained systems.
So how much it costs ?
It costs in privacy! Visa probably knows better than the CEO of Internet based company how looks their cache flow! So does your telephony, electricity companies and your ISP. The post office and shipping companies knows almost everything about your business activity if you sell goods. They know when your business becomes bigger, when you have trouble, they even know when your company is on vacation (and only a few people left).
So we pay with privacy to have an unlimited compute power, storage, network, etc. and have almost infinite computers to run your small business on.
Is it for real ?
Monday, December 24, 2007
Monday, March 5, 2007
IPS/IDS and grid
Last week I sent an e-mail to one of the mailing lists that I am on about IPS and IDS systems. The difference is clear, IPS can prevent events that IDS detects. Those systems are in use for long time now. If I may add - unsuccessfully since after you have those systems installed you are not more secure.
You can get information about some threats - but what is is good for ? Probably for some cases where you can get the info about an attacker. Actually you can get more information but it still does not make you or your site more secure.
The problem is that they are not reliable. It is not due to programmer fault. It is the way those system works and the way of thinking! Up until now, the security personnel usually come from networking back ground. Since most of security threats come through the net, it is reasonably to ask the net people to take care of it.
There are (2) problems that I find in this concept:
- Network is only the medium and not the problem and if one agree to this statement, the medium can help to solve the problem but the problem should be found in an other place.
- No reasonable solution can be found without understanding the application protocol. Since a crook does not look like one, and if he does, it would have been easy to detect them. This exactly the same here. The information that goes from one point to an other looks legitimate. There are just a few cases (usually they are already exist in various formats for FW) where one can define an exact format of an intrusion packet.
IPS/IDS and grids
Since the grid is based on external users that use local resources, access from the net is natural. But - Is there a way identify intrusion process ?
Well - I think the answer is YES. It should be "Application based IPS/IDS". We are familiar with application firewalls which can (partially) analyze application data and block abnormal behaviour.
For grids we will need a firewall that can check grid abnormalities. It will probably monitor the following components:
- Compute Element - to check that the jobs submitted are valid. We may soon find out that a wrapper is needed protect the system.
- WMS (or RB) - the components that are looking for best matching sites for a job to run in and submit it to that site. There are additional functionality like "rewriting" jdl (which is the general job description language to the local batch queue system). In those cases it can start with a wrapper that can check that no harm will be caused.
Now it is time to design such system....
You can get information about some threats - but what is is good for ? Probably for some cases where you can get the info about an attacker. Actually you can get more information but it still does not make you or your site more secure.
The problem is that they are not reliable. It is not due to programmer fault. It is the way those system works and the way of thinking! Up until now, the security personnel usually come from networking back ground. Since most of security threats come through the net, it is reasonably to ask the net people to take care of it.
There are (2) problems that I find in this concept:
- Network is only the medium and not the problem and if one agree to this statement, the medium can help to solve the problem but the problem should be found in an other place.
- No reasonable solution can be found without understanding the application protocol. Since a crook does not look like one, and if he does, it would have been easy to detect them. This exactly the same here. The information that goes from one point to an other looks legitimate. There are just a few cases (usually they are already exist in various formats for FW) where one can define an exact format of an intrusion packet.
IPS/IDS and grids
Since the grid is based on external users that use local resources, access from the net is natural. But - Is there a way identify intrusion process ?
Well - I think the answer is YES. It should be "Application based IPS/IDS". We are familiar with application firewalls which can (partially) analyze application data and block abnormal behaviour.
For grids we will need a firewall that can check grid abnormalities. It will probably monitor the following components:
- Compute Element - to check that the jobs submitted are valid. We may soon find out that a wrapper is needed protect the system.
- WMS (or RB) - the components that are looking for best matching sites for a job to run in and submit it to that site. There are additional functionality like "rewriting" jdl (which is the general job description language to the local batch queue system). In those cases it can start with a wrapper that can check that no harm will be caused.
Now it is time to design such system....
Sunday, February 25, 2007
What makes a Grid ?
Once in a while, the question of what is a grid is raised. One attitude is that if you can sell it as grid - then it is a grid. This is done by some of the software companies. In fact, it does not matter at all what is a grid.
But since mathematicians (and computer scientists) need definitions, I prefer to use Ian Fosters definition taken from What is the Grid ? :
(1) Coordinates resources that are not subject to centralized control
(2) Using standard, open, general-purpose protocols and interfaces
(3) Delivers nontrivial qualities of service
This definition actually requests a group of clusters that each belong to an other sys-admin (that are not under the same management to achieve non centralized control) that share resources based on open-standard protocols (as Open Grid Forum).
One of the implementation based on Globus is the EU project named EGEE-II. The project added or designed some of the components different of the original GGf (ancestor of OGF) like Resource Broker (RB) which is the node that gets the job and looks for the most appropriate site to send the job to. After it finds a site, the job is translated for the site queuing system (which is called LRMS).
In order to submit a job, one has to have X.509 certificate approved by a known CA. The user ID is translated in each site according to the local gridmap file (where each certificate DN appears). There is a pool of users that each user is mapped into according to users virtual organization belonging. A dynamic system named Virtual Organization Management System (VOMS) was developed that brings (2) main benefits:
(1) Users can have roles in the VO (opposed to flat mapping in the gridmap file).
(2) same certificate DN can belong to more than one VO. Especially in the academic environment, a user mighr work in more than one project that share different resources.
But since mathematicians (and computer scientists) need definitions, I prefer to use Ian Fosters definition taken from What is the Grid ? :
(1) Coordinates resources that are not subject to centralized control
(2) Using standard, open, general-purpose protocols and interfaces
(3) Delivers nontrivial qualities of service
This definition actually requests a group of clusters that each belong to an other sys-admin (that are not under the same management to achieve non centralized control) that share resources based on open-standard protocols (as Open Grid Forum).
One of the implementation based on Globus is the EU project named EGEE-II. The project added or designed some of the components different of the original GGf (ancestor of OGF) like Resource Broker (RB) which is the node that gets the job and looks for the most appropriate site to send the job to. After it finds a site, the job is translated for the site queuing system (which is called LRMS).
In order to submit a job, one has to have X.509 certificate approved by a known CA. The user ID is translated in each site according to the local gridmap file (where each certificate DN appears). There is a pool of users that each user is mapped into according to users virtual organization belonging. A dynamic system named Virtual Organization Management System (VOMS) was developed that brings (2) main benefits:
(1) Users can have roles in the VO (opposed to flat mapping in the gridmap file).
(2) same certificate DN can belong to more than one VO. Especially in the academic environment, a user mighr work in more than one project that share different resources.
Thursday, February 22, 2007
What makes a new technology ready for implementation ?
Both technologies that I deal with those days - grids and IPv6 are not young (more than ten years), have clear benefit for the users, are supported and implemented by the major manufactures in that field, but still failed to be adopted in large scale. I wonder what makes a technology "ready to be adopted" and how it can be identified.
There are several aspects of readiness. The first one would be "How it can be done (based on existing tools) ?".
The DoD published the Technology Readiness Assessment (TRA) Deskbook that deals with technology maturity estimation based on analytical measurements of applying those technologies. So if the different components that are needed exist or can be developed with existing technology, that makes it ready. It is probably correct from the technical point of view even though not all connections and dependencies between components can be identified.
Next one would probably be the rentability of a technology. Its actual cost vs. profit it can give. If it costs more than its revenue - no one would implement it, but if it offers significant benefit compared to the cost, then implementing it would contribute to whom adopts the technology. Since the cost and revenue measurements are not global terms (it can give different revenue for different people) same technology might be good for one and not good enough for others.
But I was looking for something else that I might call it "coolness" or the trendiness of a technology. When a technology become trendy ? It is clear that first a technology has to exist, which means it passed the initial vision, brainstorming, planning, etc. stages. Then it has to have some prototype and from this stage it can become a trend if it has a specific property. I am not sure if there is one property or several properties and weather the properties have to exist all together or a sub-set of them are enough.
Assuming there are some properties that are needed, how can a good technology can be given those properties in order to make it happen. What comes first the technology worthiness or the properties that makes it worthy?
On the other hand are there technologies that can not become trendy ? It is clear to me that it is not about being useful, not about being cheap and not about being user friendly it is about something else that I try to find out.
What is that property and how can it be identified ?
There are several aspects of readiness. The first one would be "How it can be done (based on existing tools) ?".
The DoD published the Technology Readiness Assessment (TRA) Deskbook that deals with technology maturity estimation based on analytical measurements of applying those technologies. So if the different components that are needed exist or can be developed with existing technology, that makes it ready. It is probably correct from the technical point of view even though not all connections and dependencies between components can be identified.
Next one would probably be the rentability of a technology. Its actual cost vs. profit it can give. If it costs more than its revenue - no one would implement it, but if it offers significant benefit compared to the cost, then implementing it would contribute to whom adopts the technology. Since the cost and revenue measurements are not global terms (it can give different revenue for different people) same technology might be good for one and not good enough for others.
But I was looking for something else that I might call it "coolness" or the trendiness of a technology. When a technology become trendy ? It is clear that first a technology has to exist, which means it passed the initial vision, brainstorming, planning, etc. stages. Then it has to have some prototype and from this stage it can become a trend if it has a specific property. I am not sure if there is one property or several properties and weather the properties have to exist all together or a sub-set of them are enough.
Assuming there are some properties that are needed, how can a good technology can be given those properties in order to make it happen. What comes first the technology worthiness or the properties that makes it worthy?
On the other hand are there technologies that can not become trendy ? It is clear to me that it is not about being useful, not about being cheap and not about being user friendly it is about something else that I try to find out.
What is that property and how can it be identified ?
Wednesday, February 21, 2007
First post
After several people asked me too many times, I decided to do it too - Wrting a blog.
This week was a very interesting week from the professional point of view.
On Sunday, I wasn invited by local branch of MS to see the new Compute Cluster Server (CCS). Since I lead the technical effort of the Grid in Israel, they wanted very much that I will see and express my opinion in that area.
Well - I am happy to see that parallel computing arrived to MS platform at last. It is clear that since there are multi core CPUs that become ubiquitous, it is unavoidable that it will be used by the general public and not only in the exact science dept at Universities. This service is just the first step and I am sure that MS push it to show they are interested in this field. It lacks most of the up-to-date technology as check-pointing, DAG process dependencies, advanced scripting options, etc. It will certainly make this field interesting in the coming years. I hope that it will not produce two parallel universes that never meet.
On Monday I gave a lecture about Grid security in the ISOC-IL annual conference. There were a lot of people there. Most of them were interested in community services. It is about time that the internet activity will be dominated by the people who use it and not by the technical staff that maintain it.
An interesting session was about IPv6. Yes - that old protocol that is still looking for a way to the real world. It is definitely the right direction. For long time a lot of talking and ink was spent on this protocol and now it is time to use it.
More to be followed
This week was a very interesting week from the professional point of view.
On Sunday, I wasn invited by local branch of MS to see the new Compute Cluster Server (CCS). Since I lead the technical effort of the Grid in Israel, they wanted very much that I will see and express my opinion in that area.
Well - I am happy to see that parallel computing arrived to MS platform at last. It is clear that since there are multi core CPUs that become ubiquitous, it is unavoidable that it will be used by the general public and not only in the exact science dept at Universities. This service is just the first step and I am sure that MS push it to show they are interested in this field. It lacks most of the up-to-date technology as check-pointing, DAG process dependencies, advanced scripting options, etc. It will certainly make this field interesting in the coming years. I hope that it will not produce two parallel universes that never meet.
On Monday I gave a lecture about Grid security in the ISOC-IL annual conference. There were a lot of people there. Most of them were interested in community services. It is about time that the internet activity will be dominated by the people who use it and not by the technical staff that maintain it.
An interesting session was about IPv6. Yes - that old protocol that is still looking for a way to the real world. It is definitely the right direction. For long time a lot of talking and ink was spent on this protocol and now it is time to use it.
More to be followed
Subscribe to:
Posts (Atom)