I'm a US military officer working in an IT career field, and we're all required to maintain some of these certifications. As in we literally can't have a privileged account to touch our own LAN unless a third-party entity like CompTIA, (ISC)2, ISACA, Cisco, or SANS deems us fit.
The whole ecosystem and culture around this is just ridiculous. Despite entering this career field with undergraduate and graduate degrees in CS from serious universities, the first (and only) question that people asked me was, "what certs do you have?" People go around talking about which "cert" they want next, and which ones are the hardest to get. People list all their "certs" in their email signatures after their name. People really measure their worth, and the worth of others, by these things.
So I've played the game by reading the exam guides and taking the tests (the military pays the $300-$1,500 testing fees for us in most cases, at least) and got "my certs." Now I'm a CISSP, CISM, CEH, and CCNA. The content of the exams isn't all bad; it's certainly different from academic CS and more applicable to managing IT. But these are just multiple-choice tests that one can easily pass with superficial knowledge obtained from a 100-page exam guide. Someone asked if CISSP was the hardest test I've ever taken... no... my final exams for Theory of Computation and Advanced Algorithms were...
Anyway, I've come to find out that most people don't even read the exam guides to learn what little material they contain. The typical approach, apparently, is to acquire a "dump," which is the bank of actual questions, and then repeatedly take "practice exams" so that they've already seen (and memorized) all of the questions that they might encounter during the actual exam. So, blatant cheating. I think this is very well known throughout the military IT community, and probably among the certification vendors too, but everyone turns a blind eye to it. If we stopped it, then most of our staff wouldn't be allowed to work on the LAN (see above), and then how would we get anything done? If the certification vendors stopped it, then the money would dry up as fewer people would take their exams.
In my experience hiring and then seeing how people perform over time I've roughly concluded that during screening / interviewing absence of a degree is not a negative signal (can be positive), but presence of certifications has consistently been a strong negative signal.
I know there are a few areas in which they are seemingly unavoidable, i.e. government, military, financial. But if I see certifications on a profile I am interested in and I believe we have a role that matches what the person is looking to do, then I will use the more unscripted part of the interview to go to first principles and test the foundations of CompSci understanding with far fewer buzzwords.
Certifications were the MOCs of their day. They were good for motivating and organizing your study of a topic. Unfortunately, while MOCs tend to stick to knowledge with proven value, certifications indoctrinate you into the ideas underlying the issuing company's technology, even if it is dubious and unproven. For instance, at the turn of the century, Sun's programming certifications were all about their Java Enterprise ideas that now we're all quite happy to have left behind. I took the Java Certified Web Developer exam way back in 2001 or so, and while it did cover fundamentals of HTTP and cookies and a little bit of the state of web security for the time, most of it was about Sun's ridiculous (in hindsight) web technologies like JSP.
The way the content of the exams came about was, Sun's super smart engineers thought really hard about questions like, "How can we build enterprise-scale software predictably and reliably in the future?" They came up with some ideas that seemed self-evidently true, turned them into a design orthodoxy, built massive technologies and frameworks to enable/enforce them, launched it with a tidal wave of marketing that is hard to imagine now, published hundreds of pounds of books, and created a system of certification exams all before getting enough experience with the technology to realize, "Gosh, we thought this would be a good way to build enterprise systems, but it really isn't." If you got a Java enterprise certification around the year 2000, it largely consisted of indoctrination into ideas that were about to be tossed into the dustbin of history.
At the time, I would have taken any job I got, and getting more Sun certifications like their Enterprise Architect certification (which meant you could explain why EBJ 1 was a really great idea) seemed like a good way to make myself more employable, but luckily for me, shortly after completing the Web Developer certification I got a job unrelated to enterprise Java, and I escaped further investment in that world.
Somewhere in here, there's a general (not-just-computer-science) lesson to be learned about certifications and hierarchies owned by private corporations vs. those owned by the people who work and have a place in them.
anyone doing consulting is pretty much required to do certs. If you want to be a Amazon/Google/Microsoft partner you have to have a pretty high number of certified employees, and you need to be a partner to get most of the work in the field.
cheating is definitely rampant. I find that when trying to find study material for exams, half the search results are exam dumps
Not really my field, so wasn't aware of this. What certs are they typically looking for partners to have?
And I do see it from their point of view: certs are a low-effort (on their part) way to weed out obvious frauds and scam businesses. They're borderline useless, but are at least indicative of effort.
For a company to be a Platinum Partner you'd need so many points, and the points are calculated by looking at how many of your people have which certs. Which certs were needed varied on which company you wanted to partner with. Microsoft and Cisco were the most common I saw.
Some very sharp networking engineers that I've worked with did hold a CCNP or higher certificate. They are not necessarily a bad indicator, but not a sure shot confirmation of ability either.
I did an MCSE in the nineties. It helped me round out my knowledge of the Microsoft products I was working with at the time. It also helped me to dig out of a career rut I was stuck in.
This was before all the bootcamps arrived but even then I wouldn't completely dismiss it.
Yea, back in the 90s MCSE was a better investment and was actually a good way to get hired. I know lots of successful IT people today who got their first 'real' IT job thanks to earning their MCSE. Today I don't think they have the same value.
The only good reason I can see for getting those certs is that it's a far easier path to remaining certified as a Microsoft Partner, and that's quite an enormous deal as far as software licenses, Azure credits, MSDN access, and other bennies.
I earned my Network+ cert last year for a DoD contracting gig. It was kind of nice to learn a few more port numbers, but wow: hours and hours of drilling cable standards and telco protocols from the 1990s, fiber optic connectors, servers to authenticate users dialing into a modem pool, and their official steps to troubleshoot a problem. This was for web development work, but it was basically the only cert they'd accept unless I wanted to learn SAP. And it was all just flash card knowledge. I still couldn't set up a RADIUS server.
In school I was very idealistic about learning: prioritizing understanding over grades---even a bit to my detriment. But forcing me to do pointless learning sure brought out my cynic.
In one Sherlock Holmes story, Watson is appalled to learn that Holmes still thinks the sun revolves around the Earth. But Holmes resents this new fact. It clutters up his well-organized memory with useless trivia. He tells Watson he can't wait to forget it as soon as possible. I'm as eager as Holmes to forget all I learned. It kind of shakes my identity to see myself thinking that way.
And it turns out the cert is only good for 3 years. Then I need to do it all over again. Or write 20 blog posts about networking and get them approved (for a fee I believe). I wonder if this explains some of the dumb tech blogs I see.
Like others here, I've always felt certs were a more negative signal than positive. Mine is on my resume for now, but I'm a little embarrassed by it---how crazy is that?
Which represents a real conundrum for job-seekers: you never know if you're talking to somebody who think certs are required, or somebody who thinks that lack of certs is required.
If you're going to continue the government contracting route, know that government agencies (not just DoD) love certs.
Have never done Network+ but I did another similar one that allowed a short (maybe 1-2 hour) refresher course to renew after the 3 years, and I didn't have to go through the whole test again.
I was also told to get CISSP for my current job. It consisted of a week-long class of a guy explaining "American Exam Culture" to us Europeans and practicing exam questions. Then made the exam, bunch of multiple-choice questions where you could guess most of them. Learned basically nothing and now they e-mail me asking for $35 or else I won't be allowed to call myself CISSP anymore, lol.
These things are a racket. Mostly for the companies making the test but below them there's also the whole ecosystem of business trainings, people selling "exam guides", private tutoring and what not. It's very close to a pyramid scheme.
That being said, my trainings from SANS where useful but wildly expensive. The most fun were trainings by Offensive Security such as OSCP. Good luck passing that one with just a few exam trainings.
I find amusing the fractal nature of the universe: we say that some problems are complex (i.e NP-Hard) because no matter the algorithm you design I can devise a problem instance that will take exponential time for your algorithm (informal explanation). Clearly designing metrics for human is even worse: no matter the metric, people will find a way not only to make it hard to measure, but to make the metric fail. Which takes me to the following strengthening of the Turing test:
A true AI will not only pass the Turing test, it will do so by cheating.
That raises the question of how long a link the URL shorteners can handle.
The HTTP specs places no limit on URL length. All the specs say is that browsers have finite limits and recommend that everything that transmits or receives URLs allows at least 8000 characters.
A bit of Googling suggests that Chrome can handle URLs of up to 2 MB, Safari 80K, and Firefox somewhere above 64K.
I can't find anything on how long a URL that the popular shorteners can handle.
The data URI spec does not define a size limit but says applications may impose their own.
Chrome - 2MB for the current document. Otherwise the limit is the in-memory storage limit for arbitrary blobs: if x64 and NOT ChromeOS or Android, then 2GB; otherwise, total_physical_memory / 5 (source).
Firefox - unlimited
IE ≥ 9 & Edge - 4GB
You also could use blob URLs, which have a 500Mb limit.
An example:
1. Alice constructs a data url with the base64 encoding of a zipped and encrypted document, submits it to her favorite URL shortener and gets a short URL back.
2. Alice transmits the short URL to Bob, perhaps as metadata in an cryptocurrency transaction.
3. Bob receives the short URL, navigates to it in a modern browser and extracts the data.
This is not anonymous unless the shortener is guaranteed anonymous, which it won't be.
I used to leave myself messages or short text snippets by requesting a nonexistent page on my website and adding the text as a GET query in the URL. I had my log autorotation set to email me logs after a day or two. So later in the week, when I wanted to retrieve the information, I could just search my email for the name of the nonexistent page, and the text would show up in the relevant access.log. It worked okay for what I needed and aside from the log setup (or grep'ing the apache logs) required no code.
I didn't see any certification bearer selling themselves as "experts", rather as a way to tell they know the basics - a way to compensate for lack of basic experience. On the other hand, with only little experience in a domain (let's say one project) there's no way you'd cover an entire curriculum, that's why for me a short experience is roughly equivalent to a certification exam. And even if you memorized a dump, that's a dump of answers to real world questions which you know now by heart, which is exactly the point of the exam. This goes for basic-level certifications. The "advanced" certifications are a different story - they must be doubled anyway with long time practical experience, so for those I see little value in cheating: if you tell me you're an advanced XYZ yet can't back that up with year(s) of project experience, sorry dear you took the exam (be it for good or from a dump) for exactly naught.
> The typical approach, apparently, is to acquire a "dump," which is the bank of actual questions, and then repeatedly take "practice exams" so that they've already seen (and memorized) all of the questions that they might encounter during the actual exam. So, blatant cheating.
Interestingly, when I decided to get a ham license, the method you describe as cheating is precisely how every single person and resource recommended preparing for the exam.
I find it fascinating that certs are seen as so important in the military. I work for a Fortune 500 company and we care nothing for certifications. We don’t even really care about education (unless you are applying for an internship or first job out of college). Real life work experience is all that matters. When I look at a resume and see a bunch of certificates listed, it makes me suspicious that they are trying to disguise lack of ability with multiple choice tests. But, that mostly comes back to not having any faith in certificates.
Yeah, but the difference I believe is the FCC publishes the entire question pools to prevent third parties from selling a "dump".
Ideally the pools are far larger than the questions you'd receive on the actual exam so trying to memorize the right answers is actually harder than engaging with the material.
These test providers could do the same thing. They should expand their test pools to be extremely large by tweaking the values in each question so that one Q becomes 10.
Prior to the federal government publishing question pools (I believe FAA does too), testing groups would send in folks to sit for the exam, memorize the questions then run outside and record what they saw. Do that enough times and the question pools emerged in their records and then they grew a big business selling that "dump".
I can't imagine managing large question pools manually, but with digital tools it's become trivial in edtech circles. Why these test providers don't end it is beyond me. (Probably because the federal "cert" game status quo is plenty lucrative.)
You could have legit indicators of ability on your resume and you'll get a technical interview anyway. People are being just as suspicious of certs as they are prior experience. No one really trusts resumes at all, so it makes cert trustworthiness kind of moot.
I disagree it's cheating though. Cheating is clearly defined by the exam condition, which are stupid (one time test rather than result evaluation but heh, it s a compromise).
We had this debate between dev in my company because the execs want to see everyone certified for the cloud provider they spent millions choosing without asking us, fine.
So myself I did it the hard way, spent 20 hours studying for this things, preparing for the exam, building little things to make sure I got it, to a point I feel really a lot less noob on the thing and actually start buying into it. I passed with flying color and it took me a quarter of the alloted time at the exam, fine.
But I have colleagues who just asked me what bank was the closest to what I saw at the exam, tried and only studied where they failed, some barely passed, some barely failed, and I insist each time it s better to have a holistic theoretical understanding rather than just a quick practice run but are they worst colleagues ? Some have kids, some are my managers, some are so humiliated to have to be revalidated as competent, they just cant enjoy it. They dont cheat, they do the absolute minimum to get a grade and move on.
And I challenge you to tell you never did that in your entire studying life. I remember a class of compilation theory when I was 19 that I studied the 5 hours before, and forgot 5 hours later to get a 45% grade that still passed my year and fine. Did I cheat? Hell no, but I was lazy and did the bare tolerable minimum.
But, should we certify people in IT like that, I think your manager and your team and your clients already know what you're worth, so I agree it s pointless.
> We had this debate between dev in my company because the execs want to see everyone certified for the cloud provider they spent millions choosing without asking us, fine.
I've heard of something similar.
One of the employee suggested that he didn't need the cert. This really didn't sit well with management. Then he explained why: his previous job was on the team building said cloud product they decided to use. That was the reason he got hired.
Ah that's cool :D Me all I could say was "but I spent years deploying in aws when you chose another one when everyone told you not to and nobody s using it because nobody needs it, not because we're not certified". I was met with "you need to get the cert by end of the year", which I got the next week, my eyes rolled the entire time. At least I got to impress people... like I say my colleagues all the time, the best bosses are those who care about useless metrics because they're easy to meet and you dont have to actually deliver anything of value to get praise and raise...
That's antithetical to how the computer security industry should work -- arbitrary certifications don't guarantee compliance or security, at best they suggest it.
To me, this feels like the same problem as OKRs in an organization. When your goal for success is a high score on the exams for a job rather than the ability to do the job, folks will optimize for the high score on the exam. Setting hard goals that determine success will determine where folks spend their time. Along those same lines, I don't see this as a problem for third party entities like CompTIA/(ISC)2/ISACA/Cisco/Etc to fix, but rather poor management and expectations of employees.
OKRs and certificates aren't bad, they just need to measure the objective value to the organization as opposed to arbitrary standards (such as you must have X certificate to work here, or increasing pay based on certs earned).
Those tests mentioned above can be useful litmus tests for folks in the recruiting space to sort through candidates, especially if they have a large number of applicants, but I worry that it excludes folks that could otherwise be very good at the job/role.
I agree completely and I think we should move to a project based way of learning.
Stupid example:
You build a rocket, fly it, write how you did it. That demonstrates a certain understanding of physics as
If I want to require an understanding of basic physics I can require people to have done a project which satisfy that requirement.
I've been doing something similar for hiring junior engineers and how well they perform creating a project is the single most important factor to forecast their work performance.
No leetcode required.
> That's antithetical to how the computer security industry should work -- arbitrary certifications don't guarantee compliance or security, at best they suggest it.
On the other hand, it is how computer security often does work. I can't count the number of times I've had to go through a listing of false-positive flags from some braindead OWASP checklist penetration/vulnerability tester.
Same goes for financial industry regulatory exams. Very little actual learning, a lot of remembering stupid stuff. It's no wonder people want to cheat.
I've also done network stuff, it's quite a lot more technical. But in the end it's still a silly thing to do multiple choice for. Debugging a network is a skill that's hard to boil down to that kind of exam, though to be fair they do show you how in the courses.
I will say that CCNA (taken in 2019) was the most legitimate of these exams. There were a handful of "simulator" questions that involved actually typing some commands into a Cisco IOS switch or router to change the configuration as appropriate. So that required a deeper understanding and forced test-takers to demonstrate what they might need to do on the job as a network technician. However, CCNA also seemed to be one of the exams for which it was easiest to find a "dump" including "practice simulators" taken straight from the real exam.
Yes, the Cisco certs seem to have a relatively good reputation. I think the higher ones even involve (or at least did) stages where the test-taker spends half a day setting something up, then an instructor breaks it and they have to fix it again. On the other hand, people there complain about the recertification requirements - it's quite a bit of effort to repeat every few years, so unless you're strictly required by an employer or customer to maintain them many let it lapse.
Yeah I actually used the CCNA simulators for a lot of things early on before things clicked. It's just very hard to examine for in a way that isn't easy to cheat and demonstrates real understanding. After all in the real world every problem has its own quirks.
> The typical approach, apparently, is to acquire a "dump," which is the bank of actual questions, and then repeatedly take "practice exams" so that they've already seen (and memorized) all of the questions that they might encounter during the actual exam. So, blatant cheating.
Children I know in UK high schools (12-18) are being "educated" solely in this manner that you describe as cheating.
I don't think you're wrong.
Basically, the "teacher" gives them past exam questions to do right from the very start of a 5 year course. They never get any reference books (nor online equivalents), they're not taught the subject matter, only taught how to answer the exam questions. It's diabolical but what passes for state education at the moment.
To clarify, I have no problem with practising on past exam questions but you need to actually educate people before you ask them to answer questions on the material.
There's a common pattern where in low-performing schools, the kids test above average at the lower levels and then fall off a cliff. This is exactly why: young kids (K-2) are great at memorizing. But if the only skill they learn is memorizing, their ability to extend this as they progress fails. It was startling how much the kids I encountered when I taught math (to high school kids and to college kids who were essentially taking junior high/high school math classes) still relied on memorization and pattern matching. I did what I could to break them of the habits but there's only so much you can do, especially in the post-secondary context where you get 45 hours of instruction time (less time for exams) in the course of a semester.
That happens in the US as well. Its because they use the test scores as a measurement which weigh heavily in decision making regarding teacher retention and promotion. Some teachers hand out the answer sheet 15min prior to a test for the kids to review.
For a different perspective on certifications, I just got my CKA cert from the Linux Foundation, and that test is legit: all simulation, no multiple choice. They use a full test cluster, backed by killer.sh, and every question involves some legitimate shell skills in addition to knowing k8s backwards and forwards. I was so impressed by the level of work that was put into the testing. May go back and get the other two certs if company will pay for it and it makes sense.
Why do vendors insist on multiple choice? Red Hat solved this problems by making a Red Hat certifications practical exams. You can’t cram from brain dumps if you actually need to understand the content.
Multiple choice isn’t necessarily an indicator of poor quality. Most IQ tests, those administered by psychiatric supervision, are multiple choice intentionally looking for deviation and alignment through repetition and measured variation provided a long enough test. One of the cert exams I took, only one of them, looked for the exact same thing as a means to identify and fail crammers.
Agreed, but it's also going to face even more resistance. I was ok spending all this time on something I knew I could pass quick, but practical ? Damn if you want to do a real test, get into a job with a 3 months probation and your rent on the line and contribute to something that makes money and you're done.
All these certs stuff they re like uni, you know all they prove is very theoretical and you must have too some genius colleagues who can code amazing things but somehow never deliver and some messy dude who iteratively build the money printing software that pays for all this theory.
I see “good certs” done by a truthful person shows me that a person can complete something, and they have the shared vocabulary to start understanding how that cert knowledge fits into their actual job.
Im always surprised at the lack of follow through I see at so many levels.
Certifications for me have always been a negative signal for people that we might hire for startups. I have no idea how that affects folks that want to work for well established companies. It seems like below CIO it might be required for most enterprise IT organizations.
I work as a consultant. I'm certified on some GCP stuff.
Work pays us to study, take the exam and even pays a bonus if we pass.
Am I smarter as a result of that exam? No. But I'm easier to sell. Some clients ask directly for certifications, others ask leading questions to our sales people I think.
Would i put my certifications on every variation of my CV for any job I apply for? Probably no.
While I'm at it: Someone else here writes about instantly rejecting people because of shibbolets like certifications.
I'm fairly smart, well liked by colleagues and very well liked by customers. However I have been dumped in screening I don't know how many times, dumped during interviews etc for something nobody told me but now seems to be gone.
Well everyone who didn't take me seriously: Thanks! I'm now a consultant working somewhere else with someone who pays better. And I am very motivated to give your competitors an edge ;-)
None of my certifications I got were required, I was hired into the enterprise IT field with no formal training. But I will say the process of studying for and taking certifications was wildly beneficial to my job. In my case, I took community college courses aligned to these programs (mostly to keep me on track) and often was able to apply things I was getting from class directly to work. I would say the certifications fill in a lot of gaps, and do give you a good baseline.
It's definitely going to be a "you get out what you put in" thing though. Folks trying to rote-memorize dumps aren't going to get anything out of it. Personally, I used practice tests to evaluate and tell me where I needed to go back and study more, and then jumped back into the material in those places.
Where do the dumps come from. Are test takers contributing to dumps from memory after they finish the exam. People were doing that with the patent bar, which also reuses some questions. Now test takers have to agree to confidentiality requirements.
Good question. These certification exams all include a non-disclosure or confidentiality agreement. In any case, I don't think this could be the result of test-takers recalling questions from memory. We're talking about seeing 100-150 questions, and spending only a minute or so with each one, and many questions involve reading a short paragraph. The "dumps" have the questions and answers verbatim. I think it must be the work of unscrupulous employees at the certification vendors or some of the testing centers.
Are test takers contributing to dumps from memory after they finish the exam?
I'm guessing no, because what could be the incentive to doing that unless there's money involved? Why would someone who passed the test and gained the cert want to dilute the value of the cert they just earned by making it easier for people further down the pyramid to pass the test?
there is absolutely money involved. The amount of people that has to get these certs is absolutely massive. There is a lot of money to be made in selling dumps
As far as I can tell these dumps come from the cert org itself- I needed a CompTIA cert and all of their official materials come with access to a bunch of practice tests.
There's a big difference between the legitimate practice tests that come with an exam guide and a "test dump." Practice tests are examples to give people a feel for the style and scope of the test, and which areas they might need to learn more about. A "test dump" lets people memorize pairs of actual (not example) questions and answers, so they can pass with no knowledge of the underlying concepts or how to apply them.
An important distinction for sure. For exams in general, IMO, practice tests/old exams from previous years allow one to simulate test conditions and improve one's ability to perform under those conditions.^1 This can be especially useful for short answer/essay type questions. It is not uncommon to see question re-use or at least similarity of questions from year to year, but a useful part of the exam preparation process, for me at least, is deriving the answers for onself.
1. Those might even include answers written by previous test-takers. Memorisation of such answers would be futile because the questions change from year to year.
It's the same problem test driven development faces. Some things can be inferred from a list of test cases (exam answers). But some things require deeper insight that no amount of test cases can provide without spending time actually thinking about the big picture.
It can if you go deeper and understand the why behind the answer. The problem is that it's possible to memorize what the right answer is without understanding the material at all.
The thing that always bugged me was that Offensive Security certifications basically have no official recognition within that system despite being difficult relative to the others (it's often on job posts for contractors though)
How are pay and benefits in the cert-worshipping world? It would be an interesting exercise grinding out the alphabet soup of certs as an alternative to endless algorithm practice.
In the US military, among both uniformed service members and federal civilians, it's still more about seniority, institutional knowledge, and security clearances than certifications (with actual skills being even less important). The pay and benefits are good (much better than average for a given location with similar education and years of experience) if you make a career of it, but you won't start as an O-4 or GS-13 after just passing a bunch of certification exams. Highly paid contractors are typically former military because of their institutional knowledge and security clearances.
The numbers that I see for "Big Tech" software engineers seem to be in a whole different universe, though. Better to keep practicing for your algorithm interviews.
We have a certification system for every skill in Australia. Are you a Probation and Parole officer? A preschool worker? An ambulance worker? Working in the food industry selling cheese? You will need a certificate for these skills. Have you been doing the job for 20 years? Sorry.
From about 2002, Australian Quality Training Framework (now reorganised as the Australian Quality Framework) deemed you unable to continue working in your area of expertise and demanded you get certification. Outward Bound instructors with 20 years experience would suddenly find that a neophyte with a certificate was deemed able to direct dangerous rock climbing experiences, while they could only look on in dread.
Certification was a turf fight between different bureaucracies: State versus Federal. Of course, the Federal Government won the battle, as they controlled the purse strings. University qualifications and technical college qualifications, all provided by state institutions, were ignored. A plumber had no need to sit a technical college exam. A federally-approved privately-run Registered Training Organisation (RTO) could provide you with the appropriate certification. It was all too easy. Hand over the money and take the certification test. The hack for a fork lift driver's certificate was to turn over the test paper: the answers were written on the back!
It should have been a national scandal. But, of course, we Aussies are too compliant. And in the end, the federal bureaucrats expanded their empires. I seem angry. And yes I am. Worthless paper certification is hiding the risks being taken with Australia’s future.
I mostly agree, and this isn't a defense of certs, but -
> You will need a certificate for these skills. Have you been doing the job for 20 years? Sorry.
I've encountered an alarming number of people with decades of "experience" who were in fact incompetent. Certs are a bad solution, but they're a bad solution to a real problem.
> Certification was a turf fight between different bureaucracies: State versus Federal. Of course, the Federal Government won the battle, as they controlled the purse strings. University qualifications and technical college qualifications, all provided by state institutions, were ignored. A plumber had no need to sit a technical college exam. A federally-approved privately-run Registered Training Organisation (RTO) could provide you with the appropriate certification. It was all too easy. Hand over the money and take the certification test. The hack for a fork lift driver's certificate was to turn over the test paper: the answers were written on the back!
Gaming education system is a huge cottage industry in India. Here's a list of "services" for sale I know of first hand.
1. Impersonation during an exam/interview.
2. Project work; end to end.
4. University degrees, including PhD end-to-end. You just need to attend a few classes.
A poor or lower-middle income family has very very few escape routes into middle income group. Educational credentials is one of the best ways. A college degree opens multiple doors; government jobs, interviews in private companies etc., So the clamour to get educational degree in India is almost unprecedented. For example, the competition to get into IITs is immense. Google "Kota tuitions" for gory details. Here's a small example[1]
India is currently undergoing the inflection point of development.
There's a _lot_ of very poor people in India, vastly more so than in a "fully developed" nation.
But at the same time, India's recent economic growth is enormous, giving a far larger percentage of the population opportunities they've never had before.
But there's enormous competition for those opportunities.
So basically it's the right moment (more like "right half century") for India to have this problem, there's so much opportunity but also so much competition, that each small step forwards academically can mean a jumping a generation ahead of the curve on economic growth.
To put it another way, you can model economic development by chucking a lump of meat to a school of starving piranhas.
The desperation is more in countries where being in poor or lower-middle income family restricts access to affordable quality schooling for kids, affordable sheltering, quality medicine or insurance. Hence you see more such desperation in under-developed or developing countries like India.
I wish I had less morals to make this myself, but how long untill GPT-3 (or 4) is used to just produce material to pass the whole university process? It could probably solve exam questions.
GPT-3 (and almost certainly also GPT-4) is not going to be capable of solving exam questions. I think you've got a radically exaggerated view of what it's capable of. It's a tool for generating roughly coherent text given a prompt.
I think you've got a radically under-exaggerated view of what it's capable of. It's fully capable of answering 98% of exam questions. It's even capable of producing working code from description, and much more.
> It's fully capable of answering 98% of exam questions.
What does this even mean? Is there some dataset of all exam questions? And to what standard is it capable of answering them? Do you have a source to substantiate any of this?
I simply tried. You can too. It gives fully correct answers deemed correct by the textbooks I took them from - and not word by word so it's not simple matching. It failed on a fusion physics scripta question - but I'm pretty sure almost nobody in the world knows that answer as it's very cutting edge stuff.
I don't believe it knew my exam questions because I translated them from Czech printed books. I tried computer science and literature questions - it got 100% on the literature section (5 questions) and 4/5 on the CS section.
I'll try again later and post the log too, but I'm on the go now.
Thanks - I missed these replies, but this is an interesting result. I'll grant you that the results in all the comments are certainly far better than I would have expected.
What sort of questions? Care to give a concrete example?
GPT-3 is certainly capable of regurgitating facts, even to the point of rewording them, and connecting a small number of facts together. But it's not capable of making logical deductions in any consistent manner.
The world needs a lot of Physical examination testing spaces that are run on open standards but which can be rented by different institutions. These physical spaces should have the most stringent rules to prevent unethical behaviour. It is difficult to depend on honor system in ultra competitive societies like India and as an Indian it has impacts honest people like me more.
Having said the above, unfortunately there are very few ways to do all round assessment of individuals in a purely objective manner. Human biases would always be there in subjective assessment and an interesting thing i see in India is that a lot of people mistake English speaking ability and communication skills for overall ability.
Yes we have/had this in India. There will always be a physical exam centre near you for various govt, University, private company recruitment exams. I have given my University, GRE in physical centres. There are cameras and proctors who roam around constantly to check for cheating. But now due to covid things have completely changed and have become remote. I saw a few interviews where candidates are blatantly cheating and its actually quite funny.
While it's true that cheating can reflect badly on otherwise well-performing students (testees?), the problem is in the score-adjusting based on group performance. If that does not happen, your score is yours only. Ensuring that different versions of tests are comparable in their "hardness" is what we should focus on.
Perhaps India is a culture where academic/certification performance is highly valued, instead of hands-on experience. However, if "cheats" end up getting all the jobs because of that, and they end up performing well in their jobs, there is really no harm[*], other than highlighting how certification is useless. If they end up performing bad, commercial interests should incentivize employers to not look for good academic/certification performance anymore, like it's done, for instance, in most IT jobs in the "West".
I've witnessed plenty of cheating in my University days, and I never felt "threatened" by "cheaters": I knew what I was in for, and if they weren't, that's their loss (and win, because we were not matched against each other). The only regret I have is that many of them ended up taking teaching jobs (I can't see how they can properly make their students excited about the art of CS), but it's because those good at CS had good industry jobs waiting for them at significant pay raise compared to academic careers (so it's not academic scores that hindered them from continuing in academia).
[*] Sure, principled cheaters (it may sound like an oxymoron, but it's not), should try to get rid of the certificate-valuation (by eg anonymously exposing how easy it is to cheat, and how irrelevant it is to their on-the-job performance) to stop this money-grabbing scheme altogether.
Universities and community colleges will typically let you use their exam rooms (with proctors) if you call ahead and schedule with them (sometimes with a fee).
Yes definitely. I've used this one [1] multiple times because it is a short distance from Chicago's central business district and they are responsive and highly competent for just a small amount of money. I think that when you look at public university systems in the USA, you'll see that they already have an internal need to offer test proctoring because they have students temporarily at one campus location but enrolled at a different one. Offering the service to the outside public is not much extra effort on their part.
If people are having to hire other people to cheat on tests like this, it makes me wonder if it's the system that's broken. Maybe tests aren't the best way to judge somebody's capabilities.
Any system will have people who aren't performing well, and some fraction of those people will be tempted to cheat. Cheating in this case doesn't provide any indication that the system is broken.
I'm currently interviewing for a mix of tech lead and equivalent staff engineering roles that more "individual contributor" focused
The ones that skew towards heads down work ask a lot of LC questions (the kind cheated on in the article).
These are that you can find the exact solution for in the time it takes you to read the question out loud for the interviewer...
-
Meanwhile ones that skew towards leadership are asking questions that you can't cheat nearly as easily on.
Having in-depth conversations with technically knowledgeable people, sure you could maybe you could get some sort of teleprompter, but the breadth and depth of knowledge being asked means you'd be hard pressed to keep up if you didn't already know the domain pretty well.
There's also a greater focus on talking about yourself, like things you've done for example. Now you can borrow someone's story, but again, it's many many times harder to deliver it convincingly than it is to deliver a LC answer that has a known optimal solution that you're already expected to follow near verbatim.
-
The difference is simple, the amount of resources you're willing to put into testing.
Places asking LC questions are doing so as a cheap filter.
Places assigning a high ranking engineer to talk to you are making an investment.
I understand the dilemma a few companies in tech face, like FAANG. There's so much demand they can't afford that in-depth approach for everyone.
But I do see a problem with how systemic the cheap filter approach has gotten. So many companies hurting for applications in the pipeline are putting up silly hoops that ironically seem to favor people who aren't that technically experienced.
I mean who's going to do better on a LC Hard with dynamic programming, the person who's spending 8 hours a day at work writing code, mentoring, doing code reviews, meeting stakeholders, etc... or the fresh grad who spends 8 hours a day running through your company's question list on LC?
I recently saw a post on Blind about a company dealing with a bad hire... they hired a CS PhD as a Senior dev only to find they were executing at the level of a Junior. Want to venture a guess as to how they managed that?
The COVID-19 pandemic has led to the closure of many physical examination testing spaces over the last 18 months. Hopefully these spaces will re-open soon
I took a few actuarial exams at center like this (USA, passed some failed others). I was pretty impressed. Cost was fair, seemed like it'd be hard to cheat. Most common exams being taken there looked like TOEFL, GRE, MCAT. I don't see why more professional certs couldn't do the same
We recently had a guy interview at our company and he had someone else do the technical part. He himself talked to the non technical people and his "colleague" talked to me and the other engineers.
He looked nothing like his linked in profile, but i gave him the benefit of the doubt since maybe it was an old picture... It wasn't. All this was over video chat..
I know one Indian friend in a renowned company who came in Canada as an International student. He completed his 2 year diploma course in IT/Technical support. After the course, he did not get the interviews due to lack of Canadian experience. To counter that, he modified his resume and added 2 fake experiences from top companies as Network Engineer. Within a week with new resume, he gathered 3 top offers in Ontario, negotiated and signed $70K per year with XYZ company as Lead Operations Specialist.
Not just in India, even in Canada, to make the top dollar and join upper middle class league, new breed of cons always game the system and use shortcuts. Once they join the class, they game the real estate market. Cycle goes on and on. They always win, rest of innocent ones who comply with rules and system are stuck behind with minimum wage job and renting single rooms, never get to ever see a dream house of their own.
> To counter that, he modified his resume and added 2 fake experiences from top companies as Network Engineer. Within a week with new resume, he gathered 3 top offers in Ontario, negotiated and signed $70K per year with XYZ company as Lead Operations Specialist.
That sounds like a failure of the companies to do a proper background check and follow up for proof of employment.
I ordered a copy of my background record and holy shit was it missing a lot of information. High schools jobs were listed, college work study jobs were listed, 15 years of employment was missing, followed by small patches of employment here and there. My credit record was only marginally better, it didn't have any employers listed of my 14 years working abroad. If it wasn't on my CV no one would know. And since some of these companies were merged, acquired, or out of business it's very hard to track down.
HR can realistically only check your last employer because anything older than a few years may as well be made up.
Well... yes and no. If it is indeed a "top company" that's still recognizable as such today, then I'd expect it to be around in some form.
I imagine HR could ask you for a proof-of-employment letter if you said you worked at Microsoft and Facebook* for a couple of years but neither showed up on a background check. I imagine they'd be even more suspicious if you additionally didn't have any references from either company.
* Feel free to replace with any other "top companies" of your choosing
No doubt you're right. I assume larger companies have a HR department that manage these things, have a formal process, and report this to the Number Line and other background services. But there are lots of boutique companies <25 employees where HR is the owner and they most likely don't subscribe to those services nor do they report into them.
> He completed his 2 year diploma course in IT/Technical support. After the course, he did not get the interviews due to lack of Canadian experience.
That's one explanation.
Truth is a degree or a visa doesn't check for employability (unlike in America where someone has to be able to secure employment to come or stay in the country).
> To counter that, he modified his resume and added 2 fake experiences from top companies as Network Engineer.
So he lied about being an engineer after doing a two year tech support degree? That would be a pretty big red flag to me (and by claiming he's a real engineer, he might get afoul of the law).
The article makes it sounds like the perpetrators used some high tech hacking to do this. In effect they probably just used some kind of normal screen sharing like msteams did the exam for the client and pressed ok.
I remember a friend who's dad knew somebody from school who had a great way to make money. He would 'guarantee' he could get somebody into one of their top 3 schools because he had inside contacts. If he was unsuccessful he would return the money as part of his guarantee (he did actually do this, it wasn't a total fraud).
The brilliance was that he did nothing. If the person got in he would keep the money. If not, he would return the money!
Reminds me of these cases that come from India. as another commenter here commented, "It is difficult to depend on honor system in ultra competitive societies like India and as an Indian it has impacts honest people like me more". These are degree mills that offer degrees for a price:
Competitive exams are evil, a serious problem in Asian countries including India. People have to survive inhumanly hard, exhausting studying regimes just to land a job which hardly uses a humble portion of what they learnt. Exams should only check if the attendant is skilled sufficiently for the degree/job. Competitive exams could have their organic place in the world as a kind of professional intellectual sports though.
The exams are hard since it is a poor country without many good university spots. There isn't much you can do about that, except increase the number of spots until the requirements to get in are no longer that high.
Edit: And the main problem with increasing the number of seats is corruption. You can't just give people money and expect them to create an institution that actually educates people. All of the processes required to make that work will have to be made and expanded etc. It takes a long time to go from corrupt poor country into a modern country where things like education exists in abundance.
>People have to survive inhumanly hard, exhausting studying regimes just to land a job which hardly uses a humble portion of what they learnt.
I always thought the purpose of these kinds of exams was to determine if you had enough work ethic to pass them. The information actually included is peripheral at best most of the time.
So many fascinating items in this case for multiple reasons:
- From an Expert who is good enough to pass an enormous variety
of IT certifications but reckless and inept to cover his tracks.
- The splashing of the suspect photos on a national newspaper
before their conviction in court. Love the note under
their photo... "Photo by special arrangement"
- The ability to easily overcome most software cheating
detection software etc...
There is a lot on criticism here of the education system
in India and the Certification industry. Although many of the points raised are relevant, lets not forget that cheating in unfortunately prevalent at the most exquisite levels of Academic research in the so called developed countries.
Random sample:
------------------------------------------
"More than 60 Fall CS50 Enrollees Faced Academic Dishonesty Charges"
"Luxembourg Prime Minister Bettel accused of massive plagiarism - Only two pages of 56-page university dissertation were plagiarism-free, magazine investigation finds"
Blame the dumb laws - it is unauthorized usage of the tester's system which is how hacking is often defined legally. It is a distasteful and a low technical knowledge equivocation of unauthorized computer access is always "hacking" but I can see why they would use it even without sensationalization from typical attempts at accessibility without mountains of exposition for amateurs mixed in.
I love Safari's battery-life preserving ability, but I hate that it doesn't support a JavaScript Allow-list (you can only nuke JS across the web or have it always-on). uBlock Origin (my favourite and solely-trusted ad blocker) doesn't support recent Safari versions. Anyone know of any good options in this scenario? Either a trustworthy and working ad-blocker for Safari - or a battery preserving browser for the Mac with an extension set like Chromium/Firefox's...
A WebKit-based browser for macOS with support for Firefox and Chrome extensions. I’ve been using it for the past month and uBlock Origin works perfectly!
I'm in the same boat with an older Firefox -- JavaScript is either on or off.
I generally keep it off while browsing for information.
When I'm doing local dev or working with trustworthy sites only, I turn it on.
After a while, the process of toggling it is almost automatic, though it still takes a few clicks and typing.
As I mentioned earlier, I've grown to avoid sites which tell me JS is required, and have found that it's mostly a good thing, because that type of design usually correlates with poor quality content, so they're really doing me a huge favor by denying me.
Thanks for pointing this out to me! I went through the HN thread and their front page and the project looks extremely promising!
I will wait for the project to be open sourced (as per their stated intention in their FAQ) just for that extra bit of peace of mind, and reluctantly suffer with Safari in the meanwhile.
I can't find any updates on the Chrome "fixes" for this. The best way to handle it is by long-pressing the back button to see a fuller history list. You can then hop to the page previous to the one that hijacked the button.
My opinion is that these tests are not to test competence, but to mitigate risk by transferring that risk elsewhere. Kind of a version of the 'No one ever got fired for buying IBM'.
If you give a user sudo privileges, and then they don't know what they're doing so they rm -rf or worse, then management puts you squarely to blame. If the user has a cert you can point to, and regs that say users with that cert may get sudo privileges, then you can point to the regs and the cert so that now the certifying organization is to blame. Or more likely the user will be blamed, malice will assumed instead of ignorance. Which brings to mind Bear's rule 11 (I'm Bear): Never attribute to malice, without investigation, what you can attribute to ignorance, as ignorance is common and malice is not.
Most of these images appear to be from a single instance of cheating from a state in India (Bihar) with relatively poor law enforcement. I imagine that in more higher stakes exams like the JEE, parents will likely not be able to get away with passing notes to their kids through the classroom window.
That is correct. In pan-India exams like the JEE and NEET, and even in high-stakes state-level exams, it would be impossible to cheat so blatantly.
That doesn't mean cheating doesn't happen, but it would require you to know someone in the examining body. And, they would be risking their careers and their liberty to help you.
Whether it's a scandal of this ilk, or the Lori Laughlin variant thereof, at some point, don't we need to consider the underlying incentive structure of the broader system rather than the specific behavior of the bad actors? If the dwindling path to social mobility runs through exams and a shrinking list of institutions, then there will be those who respond to these incentives unethically. How do we change the incentives?
Just like drugs, this is unsolvable because there's just so much money to be had without basically banning foreign students from attending American universities.
I have a close friend who used to get paid $5k-$10k per (Chinese) student to take tests fore them until she started looking too old to pass for a student. When she aged out she just started a "Test Prep Center", and now has about a dozen test takers working for her. She spent more $$ on her house in Palo Alto than I will make in my career.
There's a part of me that thinks it would have been nice to do college again after I finished the first time but get paid to take someone else's classes (except for the moral issues).
The whole ecosystem and culture around this is just ridiculous. Despite entering this career field with undergraduate and graduate degrees in CS from serious universities, the first (and only) question that people asked me was, "what certs do you have?" People go around talking about which "cert" they want next, and which ones are the hardest to get. People list all their "certs" in their email signatures after their name. People really measure their worth, and the worth of others, by these things.
So I've played the game by reading the exam guides and taking the tests (the military pays the $300-$1,500 testing fees for us in most cases, at least) and got "my certs." Now I'm a CISSP, CISM, CEH, and CCNA. The content of the exams isn't all bad; it's certainly different from academic CS and more applicable to managing IT. But these are just multiple-choice tests that one can easily pass with superficial knowledge obtained from a 100-page exam guide. Someone asked if CISSP was the hardest test I've ever taken... no... my final exams for Theory of Computation and Advanced Algorithms were...
Anyway, I've come to find out that most people don't even read the exam guides to learn what little material they contain. The typical approach, apparently, is to acquire a "dump," which is the bank of actual questions, and then repeatedly take "practice exams" so that they've already seen (and memorized) all of the questions that they might encounter during the actual exam. So, blatant cheating. I think this is very well known throughout the military IT community, and probably among the certification vendors too, but everyone turns a blind eye to it. If we stopped it, then most of our staff wouldn't be allowed to work on the LAN (see above), and then how would we get anything done? If the certification vendors stopped it, then the money would dry up as fewer people would take their exams.