Interviewing ChatGPT for a job

Re­cent­ly, I was re­view­ing the in­ter­view ques­tions for a few com­pa­nies. Many of them were un­for­tu­nate­ly only about al­go­rithms—sad­ly in­deed, since those ques­tions were for the jobs that have noth­ing to do with al­go­rithms, and if the in­ter­view process is eval­u­at­ing the skills that are not need­ed for the job, and not eval­u­at­ing the skills that are ac­tu­al­ly need­ed, well, bad things will hap­pen.

Re­view­ing those ques­tions, it made me won­der­ing how Chat­G­PT would solve them: my the­o­ry was that it would score pret­ty high. A lit­tle ex­per­i­ment con­firmed that. Then I start­ed to think about the ques­tions I ask to the can­di­dates dur­ing the in­ter­views, and what if I asked those ques­tions to the ar­ti­fi­cial in­tel­li­gence. Chat­G­PT sur­prised me in that it was giv­ing very com­plete re­spons­es, out­per­form­ing me (and the can­di­dates I had in the past) more than once. Some­times, how­ev­er, it wasn’t “think­ing” large­ly enough. Here’s one of the ques­tions I asked:

Imag­ine a bank­ing ap­pli­ca­tion that stores is­suers—the iden­ti­fiers of com­pa­nies. Each is­suer has six sym­bols, and each sym­bol is ei­ther a cap­i­tal let­ter, or a dig­it (so there are thir­ty-six pos­si­ble val­ues for each of the six sym­bols). You can store it in a form of a char(6) in a re­la­tion­al data­base, but your col­league tells that this wouldn’t be op­ti­mal in terms of space us­age, and sug­gests an op­ti­miza­tion. Af­ter all, every com­bi­na­tion—and there are 2,176,782,336 pos­si­ble val­ues—can be stored in 32 bits, where­as char(6) is 48 bits. What would you re­ply to your col­league?

I asked this ques­tion to a hu­man can­di­date once. The re­sponse was miss­ing the point. The can­di­date start­ed to search for a way to do such stor­age op­ti­miza­tion, and then lost the track of his own thoughts—I would be lost too if I were him. What he missed is that I was eval­u­at­ing his prag­ma­tism. The can­di­date could have an­swered that this is a sort of pre­ma­ture op­ti­miza­tion, and, more­over, it doesn’t look like there would be any no­tice­able im­pact on the data­base here. Mak­ing any com­pli­cat­ed changes in the for­mat of the stor­age would only harm the pro­ject, as it would make the code more com­plex, with no clear ben­e­fit. Fi­nal­ly, the can­di­date could be men­tion­ing the fact that data­bas­es usu­al­ly pro­vide trans­par­ent com­pres­sion that can be en­abled for a giv­en col­umn.

Chat­G­PT an­swered this:

Your col­league’s sug­ges­tion to use a more com­pact rep­re­sen­ta­tion for the is­suers in the bank­ing ap­pli­ca­tion is valid in terms of space op­ti­miza­tion. Stor­ing the is­suers us­ing 32 bits in­stead of 48 bits can save stor­age space, es­pe­cial­ly when deal­ing with a large num­ber of records. How­ev­er, there are oth­er fac­tors to con­sid­er be­fore de­cid­ing on the most ap­pro­pri­ate ap­proach. Here are some points to ad­dress in your re­ply:

1. Read­abil­i­ty and Main­tain­abil­i­ty: Stor­ing the is­suer as a char(6) pro­vides hu­man-read­able and self-ex­plana­to­ry in­for­ma­tion about the is­suer. De­vel­op­ers and data­base ad­min­is­tra­tors can eas­i­ly un­der­stand and work with the data, as the char­ac­ters rep­re­sent let­ters and dig­its. Us­ing a com­pact bi­na­ry rep­re­sen­ta­tion may sac­ri­fice read­abil­i­ty and make it more chal­leng­ing to in­ter­pret the data at a glance.

2. In­dex­ing and Query Per­for­mance: If the is­suers are fre­quent­ly used in queries, hav­ing a com­pact bi­na­ry rep­re­sen­ta­tion could im­pact in­dex­ing and query per­for­mance. With a char(6), the data­base can uti­lize in­dex­ing struc­tures ef­fi­cient­ly, mak­ing it eas­i­er to search, sort, and fil­ter by is­suer val­ues. Bi­na­ry rep­re­sen­ta­tions would re­quire ad­di­tion­al con­ver­sions or bit­wise op­er­a­tions to per­form such op­er­a­tions, po­ten­tial­ly af­fect­ing per­for­mance.

3. Ap­pli­ca­tion Log­ic and In­te­gra­tion: Con­sid­er the im­pact on the ap­pli­ca­tion log­ic and in­te­gra­tion points. If oth­er sys­tems or com­po­nents in­ter­act with the bank­ing ap­pli­ca­tion, they may ex­pect and de­pend on the char(6) rep­re­sen­ta­tion. Chang­ing the rep­re­sen­ta­tion to a com­pact bi­na­ry form would re­quire mod­i­fi­ca­tions in var­i­ous parts of the ap­pli­ca­tion and any in­te­gra­tions, po­ten­tial­ly in­tro­duc­ing com­plex­i­ty and com­pat­i­bil­i­ty is­sues.

4. Scal­a­bil­i­ty and Fu­ture Re­quire­ments: As­sess the scal­a­bil­i­ty and po­ten­tial fu­ture re­quire­ments of the bank­ing ap­pli­ca­tion. While the space op­ti­miza­tion gained from us­ing a com­pact bi­na­ry rep­re­sen­ta­tion might be ben­e­fi­cial in the short term, it’s es­sen­tial to con­sid­er fu­ture needs. If the ap­pli­ca­tion is ex­pect­ed to evolve, ac­com­mo­date ad­di­tion­al fea­tures, or ex­pand the is­suer data with ad­di­tion­al at­trib­ut­es, the com­pact rep­re­sen­ta­tion might be­come less ad­van­ta­geous over time.

5. Stor­age Costs: Eval­u­ate the over­all im­pact of stor­age costs in the con­text of the bank­ing ap­pli­ca­tion. While the dif­fer­ence in space us­age be­tween char(6) and a com­pact bi­na­ry rep­re­sen­ta­tion might seem sig­nif­i­cant on an in­di­vid­ual record lev­el, the ac­tu­al im­pact on stor­age costs de­pends on the scale of the ap­pli­ca­tion, the num­ber of records, and the stor­age in­fra­struc­ture in place. In some cas­es, the po­ten­tial sav­ings in stor­age costs may not jus­ti­fy the trade-offs men­tioned above.

In con­clu­sion, while the idea of us­ing a com­pact bi­na­ry rep­re­sen­ta­tion for stor­ing is­suers in the bank­ing ap­pli­ca­tion is in­trigu­ing in terms of space op­ti­miza­tion, it’s im­por­tant to con­sid­er fac­tors such as read­abil­i­ty, main­tain­abil­i­ty, query per­for­mance, ap­pli­ca­tion log­ic, scal­a­bil­i­ty, and stor­age costs. It would be worth­while to have a dis­cus­sion with your col­league, tak­ing these as­pects into ac­count, and make a de­ci­sion that aligns with the spe­cif­ic needs and pri­or­i­ties of the bank­ing ap­pli­ca­tion.

While pedan­ti­cal­ly ver­bose, the an­swer haven’t men­tioned trans­par­ent com­pres­sion, but re­mained fo­cused on the nar­row con­text of dif­fer­ent rep­re­sen­ta­tions of the same data. This sort of “tun­nel vi­sion” be­came much more ap­par­ent with an­oth­er ques­tion:

Imag­ine your client wants to launch a new pro­ject, and wants to know if it would be bet­ter to use Post­greSQL or Mon­goDB. What should be tak­en in con­sid­er­a­tion to make the de­ci­sion?

This ques­tion looks sim­ple, but it’s ac­tu­al­ly tricky. There are es­sen­tial­ly three pos­si­ble an­swers, cre­at­ing three groups of can­di­dates.

The first group would start ex­plain­ing that Mon­goDB is bet­ter. Most can­di­dates I in­ter­view, es­pe­cial­ly for pro­gram­mer jobs, are in this cat­e­go­ry. They have a se­ries of idées reçues about the data­bas­es, one of them be­ing that NoSQL is in­her­ent­ly bet­ter than “old lega­cy stuff,” and that it is in­her­ent­ly scal­able, by op­po­si­tion to Or­a­cle or to Mi­crosoft SQL Serv­er or to vir­tu­al­ly any data­base that has SQL in its name, that, ac­cord­ing to them, can­not scale.

The sec­ond group has a good grasp of re­la­tion­al data­bas­es and oth­er types of data­bas­es as well. They can pro­vide a pret­ty ob­jec­tive com­par­i­son of pros and cons, and they put the em­pha­sis—when an­swer­ing this par­tic­u­lar ques­tion—on the data struc­ture that is be­ing used by the ap­pli­ca­tion, i.e. can it be rep­re­sent­ed as a se­ries of ta­bles, or as a se­ries of doc­u­ments. Chat­G­PT is in this cat­e­go­ry. It talked about data mod­el, scal­a­bil­i­ty, per­for­mance, schema flex­i­bil­i­ty, ecosys­tem and tool­ing (I’m im­pressed!), ACID, de­ploy­ment and op­er­a­tions, com­mu­ni­ty and sup­port, and fi­nal­ly cost con­sid­er­a­tions. With all this in­for­ma­tion, Chat­G­PT out­per­forms any can­di­date I had un­til now. But there is ac­tu­al­ly a third group...

The third group doesn’t think about the prob­lem “the right way.” In fact, peo­ple from this cat­e­go­ry think dif­fer­ent­ly, and es­pe­cial­ly they think out­side the scope. While they can do an ac­tu­al com­par­i­son, like the peo­ple from the sec­ond cat­e­go­ry, they can also re­spond with a ques­tion. Such as, why do we need to choose a data­base in the first place. And that’s the point: maybe we do, or maybe we don’t. Maybe the ap­pli­ca­tion is struc­tured in a way that it is not fit­ted for Post­greSQL, but also not fit­ted for Mon­goDB. Maybe it should use both—part of data is ide­al­ly struc­tured as a se­ries of ta­bles, and an­oth­er part—as a se­ries of doc­u­ments. Or maybe none of the two op­tions fit. I can think of at least two ex­am­ples where this is the case: ei­ther the pro­ject doesn’t de­serve a data­base at all (and I did have pro­jects that were stor­ing data in plain files, and it worked ex­cep­tion­al­ly well), or it needs some­thing more spe­cif­ic, such as In­fluxDB.

What’s so valu­able with the per­sons from the third cat­e­go­ry is that they bring an in­sight that could com­plete­ly change the pic­ture, and save a lot of mon­ey. Imag­ine the case where, in­deed, In­fluxDB is the an­swer. But it wasn’t con­sid­ered. Why was it missed? Easy. The pro­ject man­ag­er had a talk with an ops guy, who sug­gest­ed Post­greSQL and Mon­goDB, as this is what ops were fa­mil­iar with. The guy wasn’t even con­sid­er­ing oth­er op­tions, be­cause this is out­side his area of ex­per­tise. The man­ag­er wasn’t con­sid­er­ing the oth­er op­tions nei­ther—that makes per­fect sense. He went to his team with a pre­con­ceived idea that there are only two op­tions, and there­fore asked a straight­for­ward ques­tion: which one, of the two, should we take. The team picked the least bad op­tion, and pro­ceed­ed with the de­vel­op­ment, spend­ing ex­tra time be­cause the choice they made was not op­ti­mal. Once the ap­pli­ca­tion was re­leased in pro­duc­tion, this choice caused it to per­form poor­ly in terms of per­for­mance, and waste ex­tra disk space, caus­ing ad­di­tion­al ex­pense to the com­pa­ny. So much waste, just be­cause right ques­tions were not asked ear­ly enough!

As I said, Chat­G­PT out­per­formed all the can­di­dates in terms of the abil­i­ty to ob­jec­tive­ly com­pare Post­greSQL to Mon­goDB. It sure­ly have read lots of pa­pers on the sub­ject, lots of ar­ti­cles, lots of on­line dis­cus­sions, and it was able to process all this in­for­ma­tion, and out­put it in an im­pres­sive form of a rel­a­tive­ly short an­swer that gets to the point. That’s great. But we, hu­mans, still have an edge over Chat­G­PT. Un­like ar­ti­fi­cial in­tel­li­gence, we could think out­side the box, and, giv­en a ques­tion, say to our­selves: “wait, that’s not the right ques­tion—I smell a rat; let’s think a bit about it.” And that is ab­solute­ly fan­tas­tic.

By the way, the orig­i­nal ques­tion that I was ask­ing to Chat­G­PT could look as it was in­ten­tion­al­ly try­ing to lead AI to a brute com­par­i­son of the data­bas­es. How­ev­er, I also tried, in a dif­fer­ent con­ver­sa­tion, to ask “What should you an­swer to your client?” in­stead, and the re­sponse was pret­ty much the same. Google Bard haven't had any more luck ei­ther, falling into the same er­ror of a com­par­i­son.