A computer programmer, under the pseudo name ‘smilings’, has written a reflective article at my1510.cn [zh] about his experience in developing a program for keyword filtering on mobile devices. The blogger's account informs readers about the nature of filtering – that which entails active monitoring of individual privacy. Below is a full translation of his reflection:
Error sending message. By Keso (CC: BY-NC-SA 2.0)
In my second year of graduation, the head of my department asked my colleague and me to design a program for keyword filtering. The program entails a keyword recognition system to distinguish whether or not the message contains certain keywords. If yes, the system would deny the delivery of the message. However, at the same time, it has to send out a successful delivery message to the sender. Later our clients demanded that we add another function. Upon blocking the messages, the system has to record the sender's number, the receiver's number and the content of the messages. I was too green then and accepted the task without any hesitation.
My colleague and I were very dedicated in our analysis, design and in testing all the functions. During the testing stage, we found out that the system would slow down the message delivery time by 30%. We worked overtime to increase its speed. In order to be more user friendly, we had designed an interface for the clients to manage the system. There, they could add or delete keywords, check the filtered messages and the sender's and receiver's messages. Very soon, we had an upgrade of the system and the client was very satisfied. Later, when I logged in for system maintenance, I found out that the system had captured all kinds of indecent, sensitive and private messages and I felt quite uneasy seeing that. But I did not think deep and forgot about the whole thing very quickly.
Later, when I had more experience and had exposure to different ideas, I started to look at things differently. When I look back at my devotion to the development of the filtering system, I feel so guilty. Sometimes I am worried whether the system that I have created has brought misfortune to others. Or if someone has had their privacy exposed because of that. If we refused the request in the first place, would things be different?
I don't have any answers for the above questions yet and I still feel uneasy about that. Even though I was not conscious of the issue at stake and did not have much choice… also even if we had rejected it, someone else would certainly have taken up the task.
I was soon proved right. Sometime ago, our product manager asked us to follow up with a client who wanted to have a similar program. I objected instantly this time. The manager was confused and asked me why. He laughed at my explanation and said: “This is the best opportunity for our product to get into the China market. The mobile market in China would give us very good return. You should be aware of that. Without this function, we could never have the chance to enter the market, it is so competitive.” When I tried to convince him to refuse, he looked at me as if I were an alien. We failed to convince each other and he handed over the job to another colleague.
When my colleague finished the program, I nominated myself for the testing. As I had anticipated, there was 25% decrease in speed when the usage of memories and CPU increased to 32%. I wrote a testing report and concluded: “Because of XXX reason, such function would occupy a lot of resource and reduce the speed. Suggest not to launch the product.” The product manager wrinkled at my report and asked: “How much effort do we need to push up the capacity to original level?” My colleague hesitated to answer. I said “the price is quite big”. As a programmer in pursuit of perfection, this is a contradictory move. However, in order to relieve myself from my moral tension, I betrayed my professional ethics even though I knew that I could not change anything.
In the face of profit making and survival in the market, we rarely take into consideration whether or not it is ethical for us to develop a product, or whether such product(s) would bring injustice to the others or ourselves. It seems quite normal for us to develop certain products in our quest for profit, but we seldom take into account that the products could turn into tools that are used to control ordinary people, including us. Even if we are aware of the problem, can we risk our job and say no? Is it inevitable that someone would take up the job when there is profit involved? If so, what would become of our world?
When we complain that the fruits of a tree are too bitter, it is because we have sprayed too much poisonous fertilizers during its flowering.